Jorge Turbica / blog · Blog ← All articles ES · Leer en español
Cloud

Hyperdrive MySQL goes GA and Cloudflare explains its 0.001% cache bug

Hyperdrive for MySQL is GA; the empty-rows cache bug is a lesson against caching client-specific bytes.

Cloudflare has taken Hyperdrive for MySQL to general availability (14 October 2026) and published the post-mortem of a cache bug that returned empty result sets about 0.001% of the time. If you run Workers against MySQL, the lesson about shared caches and protocol capabilities is useful even if you never touch Hyperdrive.

What changes

Hyperdrive is a distributed connection pooler that connects Workers to SQL databases: it keeps connections open and shares them across requests, which matters because opening one takes at least five network round trips. It can also cache common queries in the global CDN cache. According to Cloudflare, the difference between the MySQL beta and GA is "20 bug fixes", and the beta's goal was to validate 99.99% availability.

Each instance has two nodes. The Client runs next to the Worker, manages the local cache (if enabled) and forwards queries to the Endpoint, which keeps the pool close to the database. Both use the CDN cache of their data center, so when they land in the same one they share a cache.

The bug: empty results only appeared with caching enabled. The Client cached the exact bytes it wrote to the Worker, already translated to that driver's capabilities, while the Endpoint always works with Hyperdrive's internal capabilities. The concrete difference was CLIENT_DEPRECATE_EOF, which mysql2 does not support and for which Hyperdrive rewrites the EOF/OK packets of the result set. There were two cases: if the Endpoint cached and the Client read, the Client answered the Worker with untranslated Hyperdrive-format bytes; if the Client cached and another node read, the bytes were translated twice, ending in a premature EOF or a hang.

The fix: always store a canonical version of the result bytes, encoded with Hyperdrive's capabilities, no matter which node writes it. After the change the empty rows disappeared and the Client connection success rate reached 99.99%.

Two MySQL gotchas also came up: utf8 is a deprecated alias of utf8mb3 (1 to 3 bytes per character), so they moved to utf8mb4 to support emoji; and COM_RESET_CONNECTION resets to the server default, not the one negotiated in the handshake (for example latin1 instead of utf8mb4), so you must set the charset again after each reset.

Why it matters in production

A 0.001% failure doesn't show up in tests or in an error-rate dashboard; you find out when a user gets an empty list that should not be empty. Two takeaways. First: if you cache a response at an intermediate point, the cache key must include everything that affects its encoding, or you store a canonical format. Storing "what I sent to the client" mixes representation and content. Second: a GA with 20 fixes over the beta is an honest maturity signal, but also a reminder that the rare case is the expensive one to find.

The article gives no pricing, so I can't say anything about cost.

What I would do

  • If I run Workers with MySQL, try Hyperdrive in staging with caching on and compare results against direct database queries.
  • Check that my connections use utf8mb4 and not the utf8 alias, and that any pool that resets connections sets the charset again.
  • Add a synthetic check for a "query that must never be empty" to catch this kind of cache regression.
  • In my own caches, store a canonical representation rather than client-specific bytes.

Sources

Jorge Turbica

Software Engineer and SRE. I write about the real challenges I run into in production.

See portfolio ↗