Ship it on purpose.

An on-call engineer who's been paged 3× this week.

The old alerting system cried wolf. Logs dropped under load, and real incidents hid in a sea of flapping alerts.

Rebuild ingestion for durability first, then make alerts earn their page. Zero false positives on a replay set — or the rule doesn't ship.

0 false positives in 1,200 evaluation ticks · 4.8× throughput.

Chose Redis Streams over Kafka: one small box runs the whole thing, operational cost stays near zero at the cost of future cluster-scale ceiling. The ingest contract stays portable if we outgrow it.

A tool my team actually trusts. The pager stopped being a joke.

A new grad reworking their resume for a specific role.

LLM resume tools quietly invent experience. You trust the edit, send it out, and it's a lie about your own career.

Every edit must be inspectable, and the agent must flag what it doesn't know. Build a 30-case eval set with 15 planted hallucinations and treat recall as the launch gate.

15 / 15 planted hallucinations caught · 2 false alarms.

A sharper critique prompt caught every planted lie but added two false alarms. Shipped it anyway — false alarms are a user annoyance; a missed hallucination is a credibility bomb.

The eval set is the product spec. Every prompt change runs it before merge.

A team shipping an LLM-backed feature with no CI signal when prompts regress.

Prompt and model changes get merged on vibes. The dashboard tells someone, three days later, that answers got worse.

Put the eval signal where code review already lives: a PR check that fails when a tracked metric drops past threshold.

Design + proxy working on OpenAI; eval worker fails a seeded regression in CI.

A Go proxy on the hot path means a two-language codebase — but it keeps p99 predictable and leaves Python for eval work where my LangGraph tooling already lives.

In active build. The point: trust in AI products should be a check, not a Slack thread.