Free as in beer
In a significant move to enhance the development of distributed applications that depend on regional databases, Cloudflare has announced that Hyperdrive will now be accessible on the free plan of Cloudflare Workers. Hyperdrive empowers developers to create high-performance global applications on Workers, seamlessly integrating with existing SQL databases. By simply providing the database connection string and utilizing existing drivers, Hyperdrive accelerates database connectivity without necessitating extensive refactoring or complex configurations.
Over the past year, Hyperdrive has emerged as an essential service for teams aiming to build applications on Workers while connecting to SQL databases. This includes Cloudflare’s own engineering teams, who have adopted Hyperdrive to link Workers with their Postgres clusters for various control-plane functions related to billing, D1, R2, and Workers KV. This experience has underscored Hyperdrive’s role as a fundamental building block, addressing a common set of challenges for which viable alternatives are scarce. Our goal is to enable every developer on Workers to connect to their preferred database with optimal performance, utilizing familiar drivers and frameworks.
Performance is a feature
To illustrate the performance enhancements Hyperdrive offers, consider a straightforward benchmark. While this example is not production-ready, it reflects a typical application scenario on the Workers platform. We will use a simple table and a popular open-source driver, postgres.js, executing a standard OLTP workload from a Worker. The origin database will be located in London, while queries will be executed from Chicago.
// This is the test table we're using
// CREATE TABLE IF NOT EXISTS test_data(userId bigint, userText text, isActive bool);
import postgres from 'postgres';
let direct_conn = '';
let hyperdrive_conn = env.HYPERDRIVE.connectionString;
async function measureLatency(connString: string) {
let beginTime = Date.now();
let sql = postgres(connString);
await sql`INSERT INTO test_data VALUES (${999}, 'lorem_ipsum', ${true})`;
await sql`SELECT userId, userText, isActive FROM test_data WHERE userId = ${999}`;
let latency = Date.now() - beginTime;
ctx.waitUntil(sql.end());
return latency;
}
let directLatency = await measureLatency(direct_conn);
let hyperdriveLatency = await measureLatency(hyperdrive_conn);The code above:
- Takes a standard database connection string and uses it to create a database connection.
- Loads a user record into the database.
- Queries all records for that user.
- Measures the time taken for these operations using both a direct connection and Hyperdrive.
When connecting directly to the origin database, this sequence of queries averages around 1200 ms. By simply replacing the connection string with env.HYPERDRIVE.connectionString, this time is reduced to 500 ms, representing nearly a 60% improvement. If Hyperdrive’s caching is enabled, allowing the SELECT query to be served from cache, the time drops further to just 320 ms. This single modification can lead to a latency reduction of almost 75%. Additionally, Hyperdrive provides secure authentication and transport, along with a connection pool to safeguard your database against overload as usage scales. You can experience this firsthand through our demo application.
A demo application comparing latencies between Hyperdrive and direct-to-database connections.
Traditional SQL databases are robust and familiar but were designed for long-running compute environments and not optimized for modern serverless applications. They typically require highly stateful connections that do not align well with the global and stateless architecture of Workers. Hyperdrive addresses this issue by maintaining database connections across Cloudflare’s network, ensuring they are ready for immediate use, caching queries for rapid access, and minimizing round trips to reduce network latency.
Staying warm in the pool
Let’s delve into the concept of database connection pooling, its operational mechanics, and the challenges it addresses. Connection poolers, including Hyperdrive, aim to minimize the overhead associated with establishing and managing database connections. Each new database connection incurs additional memory and CPU usage on the database server, which can become a bottleneck as concurrent connections increase. Thus, the question arises: how should database connections be efficiently shared among clients?
There are three commonly employed strategies for connection pooling:
- Session mode: Each client is assigned a dedicated connection until they disconnect, simplifying implementation but reducing concurrency.
- Transaction mode: Clients are assigned a connection only when they are ready to execute a query or transaction, returning it to the pool once completed.
- Statement mode: Similar to transaction mode, but connections are allocated and returned for each individual statement, disallowing multi-statement transactions.
In developing Hyperdrive, we opted for the transaction-mode pooling approach. This choice strikes a balance between performance and features, accommodating a high number of short-lived clients while preserving the transactional semantics that make relational databases appealing.
Hyperdrive draws inspiration from established connection poolers, managing operations to allow users to concentrate on designing their full-stack applications. It implements limits on the number of connections dispensed, controls how long connections remain idle before being released, and manages prepared statements shared across pooled connections. These considerations ensure the origin database operates smoothly, as detailed in our documentation.
Round and round we go
The question arises: why create Hyperdrive when existing poolers already address these challenges? The answer lies in the inherent latency and round trips associated with connecting to regional poolers from Workers, mirroring the issues faced when connecting to regional databases. Establishing a connection necessitates multiple exchanges between the client and server, particularly evident in the PostgreSQL connection protocol.
Establishing a new connection to a database typically requires a minimum of five round trips, which can quickly escalate. For instance, if the latency for each round trip is approximately 125 ms from Chicago to London, the total time to establish a connection could reach 625 ms—an impractical duration for a distributed serverless environment. Hyperdrive addresses this challenge by effectively conducting these round trips over shorter distances, thereby significantly reducing latency.
Impersonating a database server
To understand how Hyperdrive achieves its performance, we must explore its internal subsystems. The first subsystem, Client, resides on the same server as the Worker and communicates directly with the database driver. Its responsibilities include caching query results and relaying queries to the Endpoint when necessary.
import postgres from "postgres";
// Connect to Hyperdrive
const sql = postgres(env.HYPERDRIVE.connectionString);
// sql will now talk over an RPC channel to Hyperdrive, instead of via TCP to PostgresOnce the connection is established, the database driver performs the expected handshake, with Client acting as a database server and sending the appropriate responses. This interaction occurs on the same Cloudflare server as the Worker, achieving a p90 latency of just 4 ms (p50 is 2 ms)—a substantial improvement over the initial 625 ms. However, the query still needs to reach the database.
In its second role, Client inspects queries from a Worker to determine if they can be served from Cloudflare’s cache. If no cached results are available, Client will reach out to the second critical subsystem, Endpoint.
In for the long haul
Understanding the Client→Endpoint connection is crucial, especially given the potential latency involved. When a cache miss occurs, Client submits a request to Cloudflare’s networking platform for a connection to the data center where Endpoint is hosted. This infrastructure maintains a pool of ready TCP connections between Cloudflare’s data centers, eliminating the need for preliminary handshakes before sending application-level traffic.
Through this TCP connection, Client sends an initialization message containing all buffered query messages from the Worker. Endpoint processes the query and streams the response back to Client, keeping the channel open for subsequent queries until Client disconnects. This design allows us to transmit queries globally without incurring unnecessary round trips.
Impersonating a database client
In addition to its role as a server, Endpoint also acts as a database client, performing the client-side handshake and processing query messages. When Endpoint needs to query the origin database, it attempts to utilize a connection from its limited pool of database connections. If an unused connection is available, it is allocated for the query, and the results are returned to Endpoint before the connection is promptly returned to the pool.
If no connections are available, Endpoint may initiate a new one, provided the pool’s connection limit has not been reached. This process mirrors the handshake performed by Client, but occurs across the network between a Cloudflare data center and the origin database. While this still involves five round trips, the distance is significantly reduced, resulting in lower latency.
Distributed choreography
As a transaction-mode pooler, Hyperdrive faces the challenge of aligning the state of the driver with the state of the connection checked out from the pool. To mitigate potential errors or corrupted results, Hyperdrive ensures that all connections are in the same state—idle and ready for a query—when they are checked out. This unique approach allows Hyperdrive to manage state across different machines without requiring shared state between Client and Endpoint.
Hyperdrive tracks prepared statements created by clients and those prepared on each origin connection in the pool. When a query comes in that expects to reuse a prepared statement, Hyperdrive verifies whether it has been prepared on the checked-out origin connection. If not, it replays the wire-protocol message sequence to prepare it on the new connection before executing the query. This meticulous management ensures that both the client and the database connection maintain synchronized expectations.
Better, faster, smarter, closer
The innovative “split connection” approach is central to Hyperdrive’s architecture, particularly in how it minimizes the time required to establish new connections. Although the same five round trips are necessary during startup, conducting them over shorter distances significantly reduces latency. The placement of Client remains unchanged, as it resides on the same server as the Worker, ensuring direct connectivity without additional network hops. The more complex aspect involves determining the optimal location for Endpoint.
Hyperdrive maintains a list of eligible data centers for hosting Endpoints, prioritizing those with sufficient capacity and optimal routing for pooled connections. However, accurately pinpointing the geographic location of a database based solely on its connection string presents a challenge. To address this, Hyperdrive employs a regional pool approach, selecting data centers based on the inferred region of the Worker connecting to Hyperdrive.
While this method was effective, it had limitations, such as the potential for selected data centers to be farther from the origin database than the requesting user. This could lead to increased latency and reduced throughput when establishing new connections. Recent improvements have transformed this system, eliminating regional pools in favor of a single global Endpoint for each Hyperdrive, strategically located as close as possible to the origin database.
The solution to accurately locating the origin database lies in the Edge Validator, which verifies connectivity at the time of Hyperdrive creation. This immediate feedback prevents users from encountering issues later on. Following successful validation, the new Placement subsystem conducts connection tests from every eligible data center to determine which can connect to the origin database the fastest. This data is then used to select the most suitable location for Endpoint, optimizing connection latency.
Serverless drivers exist, though?
While Hyperdrive employs a distinct approach, it is worth noting that other solutions, such as custom database drivers known as “serverless drivers,” have emerged to tackle similar challenges. These drivers focus on reducing round trips and expediting connections while maintaining traditional client-to-database interactions. However, Hyperdrive’s design prioritizes supporting the existing vibrant ecosystem of Postgres drivers, fostering familiarity and shared knowledge across projects.
Additionally, Hyperdrive acts as a cache for individual queries, leveraging Cloudflare’s extensive caching capabilities. This strategic positioning allows Hyperdrive to efficiently manage query results, optimizing performance for distributed users. By utilizing Cloudflare’s network, Hyperdrive can effectively address the challenges of caching and connection management.
Pick your favorite cache pun
Hyperdrive’s architecture incorporates Cloudflare’s cache, which exists in each data center. By default, these caches are separate instances, with Client and Endpoint operating independently. Recent architectural changes have enabled Client to buffer protocol messages and serve results from its cache, reducing query latencies significantly. In cases where Client cannot serve cached results, Endpoint may still provide cached results from its own cache, ensuring efficient query handling.
This dual-layer caching strategy minimizes load on the origin database while maximizing response times for follow-up queries from the same Client data center. The result is a streamlined experience for developers, enhancing the overall performance of applications built on Hyperdrive.
Come on in, the water’s fine!
With the introduction of a Free Plan for Hyperdrive and a deeper understanding of its inner workings, developers are encouraged to explore its capabilities for their next projects. Getting started is as simple as executing a single Wrangler command or using the dashboard:
wrangler hyperdrive create postgres-hyperdrive
--connection-string="postgres://user:[email protected]:5432/defaultdb"Additionally, a Deploy to Cloudflare button is available below, allowing users to quickly set up a sample Worker application using Hyperdrive with their existing Postgres database. For any inquiries or suggestions for future enhancements, feel free to join our Discord channel!
