Topic
How real systems are designed, where they break and why. Case studies and patterns.
6 posts
Build log
Every minute, every open order against every rider on duty, solved as one vehicle routing problem with VROOM. How the round works, why the solver's answer is only a proposal, and why the legs are diffed instead of rebuilt.
17 min read
Build log
One search box, six indexes, one round trip. How I built a multi-vertical search in Go on Elasticsearch, why it rebuilds its indexes instead of syncing them, and how I would sync them at a million documents per vertical.
16 min read
Build log
at and cron as a service, in Go: six services, MySQL as the queue, Cassandra for history. The decisions behind it, their trade-offs, and what load-testing it on an AWS-shaped bench found, including a bug of mine.
onSystem Design postsDatabase Internals postsGo posts
11 min read · Part 1 of Building a distributed job scheduler
Build log
An EKS and RDS lookalike on one PC, a cron platform with 30,000 jobs due at 09:00:00, and ten tests. Nothing was lost; the 300 jobs/s ceiling turned out to be an N+1 of mine.
onSystem Design postsDatabase Internals postsGo posts
11 min read · Part 3 of Building a distributed job scheduler
Build log
Why the jobs table is the queue, why MySQL and not Postgres, why Cassandra only for history, and how leases, fencing tokens and idempotency keys keep at-least-once honest. With the alternatives I rejected and the SQL behind each decision.
onSystem Design postsDatabase Internals postsGo posts
12 min read · Part 2 of Building a distributed job scheduler
Deep dive
When a hot cache key expires, N concurrent requests all miss and slam the same expensive query at once. golang.org/x/sync/singleflight collapses them into a single execution. Here's the problem it solves and exactly how it works inside.
8 min read