transaction processing

Tech Optimizer
October 1, 2025
Databricks has acquired Mooncake Labs, a database startup, to enhance its Lakebase platform. The financial details of the acquisition are undisclosed. This follows Databricks' earlier acquisition of Neon, a PostgreSQL-based cloud database platform, which has expanded its capabilities in open-source databases. Mooncake's technology allows for real-time mirroring of changes from PostgreSQL to the lakehouse, eliminating the need for complex ETL processes. Databricks has raised nearly billion in funding, including 0 million in December 2024, and has made several strategic acquisitions to strengthen its AI development environment. Founders Zhou Sun, Cheng Chen, and Pranav Aurora from Mooncake have joined Databricks as part of the acquisition.
Tech Optimizer
September 5, 2025
The introduction of Asynchronous I/O via the 'io_uring' interface in Linux enhances I/O performance, particularly benefiting Online Transaction Processing (OLTP) systems and setups with network-attached storage. This technology allows database workers to issue multiple I/O commands simultaneously, improving efficiency by eliminating idle time. Currently, Asynchronous I/O mainly supports disk-heavy read operations, with ongoing development aimed at improving write capabilities, particularly for bulk and checkpoint writes, which are essential for write-intensive OLTP workloads found in applications like vehicle telemetry, social media, and online gaming.
Tech Optimizer
September 4, 2025
PostgreSQL 18 is set to be released in September and will enhance analytics capabilities and distributed system architectures. Recent data shows PostgreSQL is the most favored database among professional developers, with 58 percent usage, up from 52 percent last year. The new version will introduce an asynchronous I/O subsystem to expedite read operations from disk, potentially improving performance by 200 to 300 percent in analytical workloads. Additionally, PostgreSQL 18 will upgrade the universally unique identifier (UUID) from version 4 to version 7 to improve database index performance in distributed computing. However, it will not include support for bitemporal queries and Property Graph Queries in SQL syntax.
Tech Optimizer
August 26, 2025
A team decided to test PostgreSQL's capabilities with a one-petabyte database, setting a challenge for December 10, with a report due by January 20. They initially sought cloud storage but found no provider could meet their requirement for a single chunk of storage, leading them to rent physical servers. They used Shardman, a distributed database engine, to manage the database across seven servers. They employed the YCSB benchmark for performance testing and faced various challenges, including data loading issues and hardware problems. By January 15, they reached 863 terabytes of data generation, falling short of their petabyte goal, and documented their findings for future reference.
Tech Optimizer
August 20, 2025
App modernization is now essential for enterprises to remain competitive, moving away from traditional systems to open cloud-native architectures, with Postgres becoming a key element of modern data strategies. Experts from AMD, EnterpriseDB, Supermicro, and Lightbits Labs discussed the importance of open standards, particularly Postgres and Kubernetes, in adapting to AI-driven transformations. High-performance block storage is crucial for low latency and high throughput in Postgres workloads, with Lightbits Labs demonstrating significant transaction capabilities. This integrated stack can lead to energy savings and improved data center efficiency. Kubernetes is also vital for enhancing application portability and scalability, with IT vendors focusing on seamless integration and lifecycle support. The demand for optimized hardware for AI applications is increasing, as enterprises seek comprehensive infrastructure solutions for deployment across various environments.
Tech Optimizer
June 21, 2025
Reynold Xin, co-founder of Databricks, highlighted the outdated nature of online transaction processing (OLTP) databases, which have not evolved significantly since the 1990s and face issues like over-provisioning and performance challenges. Databricks is introducing Lakebase, a product that separates compute from storage to enhance the efficiency of transactional databases, particularly for AI applications. Lakebase allows for instantaneous branching of databases, significantly improving workflow efficiency. Built on open-source Postgres, it supports various open storage formats and offers a copy-on-write capability to reduce costs. The separation of compute and storage is essential as streaming data becomes more integral to enterprises, enabling scalability and timely insights. Databricks aims to manage the entire data lifecycle, ensuring data remains within its ecosystem for rapid reporting and analytics. The integration of Lakebase with existing infrastructure enhances developer experience and operational maturity. The architecture supports extensive experimentation at minimal cost, fostering innovation. As AI agents become more prevalent, the focus on data evaluation and reliability will grow, necessitating a deeper examination of model accuracy.
Tech Optimizer
June 12, 2025
Databricks has launched Lakebase, a fully managed Postgres database designed for AI applications, currently in Public Preview. It integrates an operational database layer into Databricks' Data Intelligence Platform, facilitating the development of data applications and AI agents in a multi-cloud environment. Lakebase uses Neon technology within a lakehouse architecture, allowing for efficient real-time data processing and scalable operations. Key features include independent scaling of compute and storage, low latency under 10 milliseconds, high concurrency over 10,000 queries per second, rapid launch times under a second, and a consumption-based payment model. It also offers data synchronization with lakehouse tables, an online feature store for machine learning, and is managed entirely by Databricks with built-in security features. During its Private Preview, Lakebase attracted participation from hundreds of enterprises across various sectors. It is supported by a partner network including Accenture, Deloitte, and others, and will receive further enhancements in the coming months.
Search