Cloudflare has published how it scales the D1 control plane to millions of databases. The driver is a change in usage: according to the post (October 13, 2026), AI-assisted development platforms create many small databases, one per user, app or preview environment, instead of one or two big clusters. As of October 8, live databases had grown 15x in a year and 65x in two.
What changes
Each D1 database is backed by a SQLite Durable Object. Cloudflare splits operations into three planes:
- Data plane: queries. They must be fast and always served at the edge.
- Control plane: the HTTP API and internal flows. It may depend on central, single-region components.
- Async plane: analytics and billing, eventually consistent.
Each database's ID and metadata are stored in Postgres (the "creation checkpoint") and the mapping is mirrored in Workers KV, which is used as a durable store and not only as a cache. On the hot path, the Edge API Worker looks the mapping up in KV, falls back to Postgres replicas through Hyperdrive only on a miss, and forwards the query to the Durable Object via an Eyeball Worker. With Worker bindings, even the lookup is skipped.
A few details worth noting:
- Creation is wrapped in a retry loop and every attempt uses a fresh ID, so two Durable Objects never claim the same database.
- Each Durable Object heartbeats the control plane every 30 seconds plus per-database jitter. It is used to reconcile state: orphans are marked deleted and KV is corrected over time.
- Storage limits (5 GB per account on the free plan, 1 TB on Workers Paid) are not computed with a scan: deltas go into a table and a job running every second aggregates them, avoiding row-lock contention on large accounts.
- Free-tier row limits come from usage events in ClickHouse; when exceeded, a Quicksilver key with an end-of-day expiry is set and Durable Objects reject queries at the edge.
Why it matters in production
Even if you never use D1, the post is a list of multi-tenant architecture decisions every SRE eventually meets:
- Idempotent retries: retrying with fresh identifiers avoids duplicated state.
- KV as a durable store: if it were only a cache, a full purge across a million databases could trigger a metastable failure.
- Cache keys: putting the database ID in the Hyperdrive cache key would crater the hit rate; with account-level keys most heartbeats hit the cache.
- Short transactions: don't hold a Postgres transaction open while a Durable Object initializes, because the connection would stay open across continents.
- Incremental aggregation instead of scans at request time.
What I would do
- Review which parts of my own multi-tenant services sit on the hot path and depend on a single region; move them out or cache them with coarse-grained keys.
- Check whether my caches are really disposable: what happens if I flush them all at once?
- Design creation flows as idempotent, with periodic reconciliation rather than trusting everything to finish.
- Load-test above expected growth, as Cloudflare recommends.
Sources
- Cloudflare, "Under the hood, off the hot path: How D1 scales to millions of databases" (2026-10-13): https://blog.cloudflare.com/d1-control-plane/