The disaggregated storage model of Lakebase Postgres offers a robust, flexible, and cost-effective platform for data management. A critical aspect of its architecture is the efficient caching of data, which ensures high throughput and low latency while utilizing an object store like S3 for data backup. This caching operates on two distinct layers: within the distributed storage, where Postgres pages are materialized to optimize both write throughput and read performance, and on the Postgres compute itself, which serves frequently accessed pages directly from DRAM for ultra-fast access. Our team has been diligently enhancing the compute-side caching, and this discussion will outline our immediate plans while reflecting on what has already been delivered to our customers.
The standard Postgres cache
Databases are notoriously demanding when it comes to DRAM (memory) usage. They primarily rely on this memory as a data cache, with the expectation that access times to rows in the cache will be measured in nanoseconds—significantly faster than even the most advanced NVMe drives. Postgres organizes its data in rows on pages, and actively accessed pages must be loaded into a memory area known as “shared buffers.” Traditionally, Postgres stores these pages using the operating system’s filesystem, which means the OS kernel also employs its own flexible page cache to facilitate caching between Postgres shared buffers and the disk. While this shared buffers plus page cache approach functions adequately, it does come with certain drawbacks and challenges.
Downsides
- Double buffering reduces the effective amount of data that can be cached on the compute. For instance, if a compute node has 4 GB of RAM and allocates 1 GB for shared buffers, the reads from disk to populate this buffer also pass through the OS page cache, resulting in a total consumption of 2 GB of RAM for just 1 GB of cached data.
- The OS page cache lacks awareness of shared buffers or Postgres internals, preventing it from making informed decisions regarding which pages to replace.
Technical challenges
- In a disaggregated storage system like Lakebase Postgres, data read from storage bypasses the OS filesystem and page cache entirely.
- Shared buffers are a static parameter, set before starting Postgres, and cannot be adjusted without rebooting the database. This presents a significant challenge for a serverless autoscaling system such as Lakebase.
- Postgres employs a separate operating system process for each active connection, meaning that increasing shared buffers—i.e., allocating more memory to Postgres—results in greater memory management overhead for the OS across all connections.
The Lakebase cache path
With this context established, we can now explore how Databricks is addressing these challenges. Our ultimate goal is to optimize the utilization of DRAM on your compute through dynamic shared buffers in Postgres that can autoscale with your workload, utilizing up to 75% of available memory. While we aim to adjust our compute platform to support autoscaling shared buffers, we are also committed to delivering sensible incremental improvements to our customers as they become available. Each incremental delivery allows us to confidently implement various components of our roadmap while providing tangible benefits to users. Thus, we began with fixed computes, as detailed in the following sections.
Larger shared buffers
Returning to the technical challenges mentioned earlier, we recognized that a disaggregated system like Lakebase does not route its reads through the standard OS filesystem and page cache. Additionally, since Postgres shared buffers are static and cannot autoscale, we devised a solution in the form of a local file cache (LFC). This layer acts as a stand-in, creating an autoscaling cache that works in conjunction with shared buffers, maximizing the amount of data cached on the compute. This pragmatic approach enabled Lakebase Postgres to launch autoscaling and has been operational on all compute since its inception.
Although presented as a single high-speed compute cache to users, the underlying architecture supports two tiers:
- Shared buffers: Postgres’s in-memory shared buffer, offering the lowest-latency access path.
- Local file cache: An expanded secondary cache residing on the compute node’s local NVMe, providing higher capacity than memory but requiring disk I/O to access a page.
Shared buffers were conservatively tuned to avoid excessive memory consumption when operating at the minimum configured compute unit (CU), with a maximum size of 1 GB for shared buffers while the LFC utilized the remainder of the total compute cache capacity (up to 75% of DRAM). Any request that results in a miss across both tiers is routed from the compute node to the distributed storage layer. However, capping shared buffers at 1 GB meant that most cache hits had to go through the slower LFC tier. While the LFC has served us well, our intention is to phase out its current form as we advance towards fully dynamic shared buffers.
| Note: Fixed computes came first |
Our initial implementation of larger shared buffers focuses on fixed-size computes, given that shared buffers are not yet dynamic. For these, we have disabled the LFC and set shared buffers to 75% of DRAM. This change is live today for fixed-size computes with CU >= 80. Removing the ~1 GB buffer cap ensures that hot pages remain in the fastest memory layer instead of cascading down to local file storage. To verify if large shared buffers are enabled for your compute, execute show shared_buffers within a Postgres connection. An 80 CU Lakebase endpoint in Databricks should return a value of 15278640. |
By retaining hot data in shared buffers rather than the OS page cache, we also mitigate the previously mentioned downsides. There is no double buffering, meaning that 1 GB of cached data consumes only 1 GB of RAM instead of 2 GB. Furthermore, because the cache resides within Postgres rather than the kernel, eviction decisions can be made with a comprehensive understanding of the database state, positioning us to implement smarter replacement policies than those available through the OS.
However, sizing shared buffers at 75% of DRAM on fixed-size computes was not merely a matter of configuration change. This challenge relates to the process-per-backend architecture.
Addressing memory and translation overhead with huge pages
Postgres utilizes a process-based structure, wherein each backend maps shared buffers into its own address space, necessitating its own page table entries—structures maintained by the kernel that facilitate the translation of virtual addresses to physical memory. By default, Linux performs this mapping using 4 KB pages. For example, each 1 GB of shared buffers corresponds to 262,144 page table entries per process. With 32 GB of shared buffers and 512 backends, this results in approximately 4.3 billion entries, or around 32 GB of page tables required to map 32 GB of cache. This working set far exceeds the capacity of the Translation Lookaside Buffer (TLB), a cache within the CPU’s memory management unit that accelerates virtual-to-physical translation. Consequently, even a cache hit incurs penalties from TLB misses and page table walks.
To alleviate this issue, the Postgres community recommends utilizing an OS mechanism called huge pages (2 MB each) in conjunction with large shared buffers. Transitioning to huge pages reduces page table sizes by a factor of 512 and significantly lowers TLB miss rates. In our benchmark tests, configuring Postgres with huge pages resulted in a reduction of tail read latency by approximately 40% and a decrease in CPU utilization by up to 30%.
Huge page support in virtualized environments
Lakebase Postgres operates within lightweight guest virtual machines on bare-metal hosts. Memory address translation involves two layers of virtualization. To effectively leverage huge pages, a consistent implementation across the entire stack is essential—from host-level reservation, through the hypervisor managing the VM’s memory, to the guest kernel. Any breakdown at any level can diminish the performance benefits achieved.
Recently, we introduced dedicated huge-page backing across our VM infrastructure, opting for explicit 2 MB HugeTLB pages rather than relying on best-effort transparent huge pages. Now, VMs allocated for large fixed-size computes initialize with a predetermined volume of huge pages sufficient for Postgres startup. To optimize system resources, compute startup automatically releases any surplus huge pages beyond those required by Postgres.
Tip: To check if large explicit huge pages are enabled for your compute, run show huge_pages within a Postgres connection. An 80 CU Lakebase endpoint should return a value of "on". |
Production results
The rollout of these enhancements began region by region a few weeks ago. Below are examples measured on large production endpoints following the restart that enabled the new configuration.
Example 1: ~2× throughput, 5× fewer reads from storage
On one large endpoint, the changes took effect around 06:10 UTC on August 11. The number of accessed Postgres blocks per second doubled, serving as a proxy for throughput. The customer reported reduced p50 and p99 latency compared to the previous day, week, and month. This endpoint had a large local file cache, and with the introduction of larger shared buffers, the storage GetPage/s decreased from about 8K per second to approximately 1.5K.
Example 2: 1.3× throughput
On another large endpoint, the changes became active around 01:30 UTC on August 14, resulting in a throughput increase of about 43%. The compute cache hit rate approached nearly 100%, with requests being served almost entirely from the shared buffers.
Example 3: 5× lower CPU use, 2× higher throughput
In this scenario, CPU utilization dropped from 20 cores to just 4 following the rollout on August 15. The compute cache hit rate surged to almost 100%, and the measured throughput doubled.
Part 2: autoscaling
We are currently focused on extending the benefits of larger shared buffers to autoscaling Postgres computes. This introduces additional complexity, as we must dynamically expand shared buffers during scaling up and reduce them when scaling down—all while ensuring the precise allocation of huge pages. To advance beyond fixed-sized computes, we have developed a protocol for autoscaling huge pages provided to the guest. These huge pages will scale in tandem with dynamic shared buffers, ensuring efficient address translation even at high concurrency and memory sizes. Our next post (part 2) will delve into the technical details of implementing dynamic shared buffers, including the current state of open-source Postgres and the areas we aim to enhance and contribute upstream.
Try it
All these performance enhancements stem from the Lakebase Postgres architecture. The storage layer serves as the authoritative system of record, while a compute node remains stateless, with its memory functioning solely as a caching layer. We invite you to deploy Lakebase Postgres and put its performance to the test. Get started here.
Lakebase Postgres can function as a standalone database, and it can also be seamlessly integrated with the broader Databricks Data + AI Platform, encompassing Unity Catalog governance, lakehouse analytics, notebooks, and AI workflows.
