Developers are increasingly using Postgres for application needs alongside lakehouse technologies for analytics and data science. Postgres has evolved into a versatile operational database suitable for various applications, while lakehouse technologies enable organizations to manage and analyze large-scale data efficiently. There is a growing synergy between these systems, although they often operate in silos managed by different teams. An emerging architectural paradigm views Postgres and lakehouses as complementary layers within a modular framework to address both operational and analytical requirements.
Historically, OLTP systems were used for transactions and OLAP systems for analysis, but this model is shifting. Scenarios such as financial applications requiring real-time risk reports and SaaS applications calculating usage metrics are becoming standard. Effective communication between product-engineering teams managing operational systems and data teams overseeing lakehouse services is essential for enhancing system resilience.
A trend called operational medallion architecture incorporates bronze, silver, and gold layers for real-time and user-facing systems. The bronze layer consists of raw data, the silver layer contains cleaned data in Postgres for real-time analytics, and the gold layer features pre-aggregated data for low-latency experiences. Each layer is queryable, allowing for bidirectional data movement between S3 and Postgres.
Key developments facilitating the integration of operational databases and lakehouses include the maturation of Iceberg as a flexible table format, the evolution of Postgres with support for various data types and querying capabilities, and a growing demand for composability among developers. The market is moving towards modular, open, and developer-friendly architectures that integrate operational and analytical layers. Future data infrastructure will focus on interconnected components and coherent modern data architecture that serves both operational and non-operational use cases seamlessly.