Aurora PostgreSQL Archives

Tech Optimizer

October 11, 2026

Amazon Aurora PostgreSQL now supports direct querying of Apache Iceberg and Parquet data in your data lake

Amazon has introduced a new capability for Amazon Aurora PostgreSQL that allows users to query operational data alongside data stored in data lakes formatted in Apache Iceberg and Apache Parquet. This feature enables direct access to Iceberg REST Catalog-compatible catalogs, eliminating the need for data duplication or relocation. The integration of DuckDB within Aurora PostgreSQL allows users to query live operational data, including uncommitted writes, alongside data from data lakes using standard PostgreSQL syntax. This capability is available on Aurora PostgreSQL versions 17 (starting with 17.11) and 18 (starting with 18.6). Users must create an Aurora PostgreSQL cluster, attach an IAM role with the AuroraAnalytics feature, and enable the aurora_analytics extension to utilize this feature. They can create foreign tables referencing Iceberg or Parquet data in the data lake, and query them using familiar PostgreSQL commands. Aurora optimizes query performance through techniques like predicate pushdown and column pruning, and users can monitor query behavior with metrics provided by aurora_analytics_stat_statements(). Direct querying of Apache Iceberg and Parquet data is available across all commercial AWS Regions and AWS GovCloud (US) Regions without additional charges, aside from incremental Aurora compute and Amazon S3 request costs.

Tech Optimizer

October 10, 2026

AWS embeds DuckDB in its PostgreSQL DBMS to speed queries of live and historical data

Amazon Web Services Inc. has enhanced its Aurora PostgreSQL database management system to allow users to query the Apache Iceberg data lake directly. This feature integrates the DuckDB analytical engine, enabling seamless access to live transactions and historical records without data duplication or complex ETL processes. Customers can use existing PostgreSQL applications to query data in Amazon S3, including S3 Tables. The integration aims to reduce the engineering workload associated with data pipelines and supports real-time applications. DuckDB processes analytical scans within Aurora, eliminating extra network hops and allowing single queries to access both live and archived data. The feature supports external catalogs compliant with the Iceberg REST Catalog specification and can create foreign tables referencing data across multiple catalogs. Aurora optimizes query performance by filtering records and caching frequently accessed data. Customers can combine recent transactions in Aurora with historical data from S3 without manual schema definitions. For low-latency workloads, selected data can be copied into native Aurora tables using standard SQL. This capability requires the aurora_analytics extension and an AWS IAM role for S3 and Glue access, and is compatible with Aurora PostgreSQL versions 17.11 and 18.6. The feature is available across all AWS regions at no additional charge, with costs incurred only for Aurora computing resources and S3 requests.

Tech Optimizer

October 10, 2026

AWS Embeds DuckDB in Aurora PostgreSQL, Enabling Direct Data Lake Queries Without ETL

Amazon Web Services (AWS) has integrated the open-source analytics engine DuckDB into its Amazon Aurora PostgreSQL database, allowing users to query Apache Iceberg tables and Parquet files directly from data lakes without traditional ETL processes. This feature, announced on September 30, 2026, enables seamless access to both operational and historical data in a single SQL statement. Users can enable the aurora_analytics extension, assign an IAM role for S3 and AWS Glue access, and create external tables using standard PostgreSQL syntax. The integration supports Iceberg tables managed by AWS Glue Data Catalog, Iceberg REST Catalog-compatible external catalogs, Parquet files on S3, and S3 Tables. Performance optimizations include predicate pushdown and column pruning on the data lake side, with caching for frequently accessed data. This feature is available for Aurora PostgreSQL versions 17.11 and later, and 18.6 and later, across all commercial regions and AWS GovCloud (US), with no additional charges beyond Aurora compute and S3 request fees. In a separate development, AWS has a significant contract with AI company Anthropic, anticipating billion in cloud-related contractual obligations over several years, with approximately billion projected to be paid to Amazon from a 10-year agreement announced in April 2026. Anthropic's revenue has shown impressive growth, with first-quarter 2026 revenue reaching 0 million and second-quarter revenue at .8 billion. However, risks are present, as a significant portion of Anthropic's revenue comes from a few customers, and many larger clients lack long-term contracts. Amazon has invested .8 billion in Anthropic, with its stake valued at approximately billion as of June 2026.

Tech Optimizer

August 18, 2026

Migrate Amazon Aurora PostgreSQL across major versions with active Debezium CDC connectors using native logical replication

Organizations using Debezium Change Data Capture (CDC) connectors on Amazon Aurora PostgreSQL-Compatible Edition face challenges during major database version upgrades. In-place upgrades require downtime, while blue/green deployments disrupt CDC event pipelines. A method using PostgreSQL's native logical replication allows migration between a source cluster (current version) and a target cluster (new major version) without a full connector re-snapshot. This approach minimizes risks of data loss and allows existing Debezium connectors to operate on the source during synchronization. The cutover process involves disabling the replication subscription, synchronizing sequences, creating a new Debezium replication slot on the target, and redirecting the connector, resulting in a brief pause for Kafka consumers. Prerequisites for this method include enabling logical replication on the source cluster, ensuring the target cluster is provisioned in the same VPC, and confirming that tables participating in CDC have primary keys. The process involves configuring parameter groups, creating publications, and establishing subscriptions for logical replication. During cutover, application writes are halted, and replication lag must be monitored to ensure it reaches zero before proceeding. After the cutover, ongoing monitoring of replication health and Kafka consumer offsets is necessary. Common failure scenarios include Aurora writer failover, Debezium connector stalls, large messages exceeding Kafka limits, DDL changes during synchronization, and long-running transactions blocking lag drain. Each scenario has specific symptoms, detection methods, and resolutions. Post-migration, organizations should establish monitoring for replication lag growth and Kafka consumer offset divergence. Automation can streamline the migration process for organizations managing multiple clusters, utilizing AWS Cloud Development Kit (CDK) constructs for cluster provisioning, schema replication, and cutover orchestration. After verifying stable operation on the target cluster, the old Debezium connector and source cluster can be decommissioned. Limitations include the absence of failover slots on Aurora PostgreSQL 17 and the need to handle TOAST columns appropriately.

Tech Optimizer

August 13, 2026

Enforcing TLS and managing certificate rotation for RDS and Amazon Aurora PostgreSQL

When an Amazon RDS certificate expires and the application’s trust store is outdated, it can cause connection failures. AWS announces Certificate Authorities (CA) rotation events in advance and provides management tools, but a lack of proactive certificate lifecycle management can lead to unplanned downtime. Default configurations for RDS and Amazon Aurora allow both encrypted and unencrypted PostgreSQL connections, which can expose sensitive data if not properly configured. AWS manages the Certificate Authority infrastructure and provisions server certificates, while users are responsible for enforcing TLS usage and managing the rotation lifecycle. Amazon RDS employs a managed Certificate Authority hierarchy that issues server certificates for each database instance, supporting TLS 1.2 and TLS 1.3. The solution to enforce TLS includes server-side enforcement with the rds.force_ssl parameter, client-side verification with the sslmode=verify-full parameter, and automated lifecycle monitoring using Amazon EventBridge, AWS Lambda, and Amazon CloudWatch. Prerequisites for implementing this solution include an AWS account with an Amazon RDS for PostgreSQL or Amazon Aurora PostgreSQL instance, IAM permissions for relevant services, a compatible PostgreSQL client, familiarity with Amazon RDS parameter groups, and the openssl CLI. The rds.force_ssl parameter controls whether Amazon RDS rejects non-SSL connections, with defaults varying by PostgreSQL version.

Tech Optimizer

August 10, 2026

Microsoft’s PostgreSQL alternative, HorizonDB: Worth the wait?

AWS's Aurora PostgreSQL service became generally available in 2017 and has evolved into an AI-ready offering with features like vector search and integrations with Amazon Bedrock. Google Cloud launched AlloyDB in 2022, which includes AlloyDB AI with vector search and model interaction. Databricks introduced Lakebase, available on AWS and Azure in 2023, while Snowflake released Snowflake Postgres in February 2026. Microsoft announced HorizonDB at Ignite in November 2025, featuring disaggregated compute and storage, a database-as-log design, native vector search, and integration with Foundry and Fabric.

Tech Optimizer

July 30, 2026

Generating Referentially Consistent Synthetic Test Data for PostgreSQL in .NET Application Modernization

Satyajit Gokhale, Aiswaryaa Venugopalan, Ruchi Agarwal, Nits Jeganathan, Khurram Khawaja, Vijay Mandadi, and Luke Huan contributed to this article. AWS Transform offers a .NET application modernization service that allows enterprise customers to migrate from Microsoft SQL Server to Amazon Aurora PostgreSQL Compatible Edition. This migration reduces licensing costs and enhances scalability while utilizing an open-source ecosystem. Schema conversion tools can assist in translating DDL to PostgreSQL-compatible definitions, but correctness must be verified by teams to ensure performance under real-world workloads. Traditional testing methods, such as using production data or manual checks, are not scalable and raise privacy concerns. Therefore, there is a need for realistic synthetic data generated at scale that adheres to the migrated schema's constraints. AWS Transform for SQL Server modernization employs an AI agent to generate constraint-respecting synthetic data at scale during the migration workflow. This process involves loading the PostgreSQL schema into a live database, constructing a dependency graph for data generation, and distributing tasks across multiple sub-agents. The agent validates each generated INSERT against the live database, ensuring compliance with constraints and providing error diagnostics for any violations. The synthetic data generation workflow consists of four stages: schema analysis, establishing generation order, distributing tasks across sub-agents, and validation with error recovery. The agent extracts metadata from the PostgreSQL schema, determines a valid table generation sequence, parallelizes data generation while maintaining referential integrity, and generates and validates data row-by-row against the live database. Each successful insertion enriches the context for subsequent generations, while errors are handled through structured diagnostics and iterative regeneration. The process ensures referential integrity and preserves the generation state, allowing for recovery from interruptions without data loss.

Tech Optimizer

July 30, 2026

EDB Says Postgres AI Tops Enterprise AI Data Platform Benchmarks

EnterpriseDB (EDB) has released benchmark results for its EDB Postgres AI (EDB PG AI), showing superior performance compared to various AI data platforms, including vector databases and managed Postgres platforms. The independent testing by McKnight Consulting Group indicated that EDB PG AI outperformed competitors in all evaluated metrics, including query speed, accuracy, and cost efficiency. Key findings include: - EDB PG AI achieved a median query latency of 50 milliseconds at 50 million vectors, making it the fastest platform tested, 80 times faster than Databricks and 21 times faster than MongoDB Atlas. - It demonstrated the highest accuracy for core vector search with a recall rate of 0.911, outperforming MongoDB by 26% and Databricks by 17%. - In a concurrent agent loop test, EDB PG AI completed tasks in 27 milliseconds, significantly faster than MongoDB Atlas (699 milliseconds) and Databricks (5.1 seconds). - EDB PG AI provided 76 times better price performance than Databricks and 34 times better than MongoDB. - The platform integrates vectors, structured data, and analytics within a single operational database, enhancing efficiency and reducing the risk of data inconsistency. The benchmark study assessed the performance of EDB PG AI against Amazon Aurora PostgreSQL, Databricks, Crunchy Bridge, MongoDB Atlas, and open source PostgreSQL, all standardized on equivalent enterprise hardware.

Tech Optimizer

July 28, 2026

Migrate self-managed PostgreSQL to Amazon RDS using the RDS console

Running PostgreSQL on Amazon EC2 requires users to manage operational tasks such as patching, backups, and security. Migrating to Amazon RDS for PostgreSQL or Amazon Aurora PostgreSQL-Compatible Edition offers a fully managed service that automates these tasks, allowing development teams to focus on application development. The migration process uses the auto-migration feature in the Amazon RDS console, supported by AWS Database Migration Service (AWS DMS), which facilitates PostgreSQL-to-PostgreSQL migrations using native tools for accurate schema mapping and faster migration. The migration can be executed using various methods, including pg_dump/pg_restore, manual AWS DMS tasks, or third-party tools, with the RDS console approach being preferred for its reliability and minimal setup effort. AWS DMS supports three replication modes: full load, CDC only, and full load + CDC, with the migration duration depending on database size and network throughput. Prerequisites for migration include having a source PostgreSQL 10.4+ on Amazon EC2, a target RDS for PostgreSQL or Aurora PostgreSQL, proper IAM roles, stored credentials in Secrets Manager, and appropriate networking configurations. Considerations include unsupported objects, potential issues with sequence values, and the need to avoid schema changes during CDC. The migration involves preparing the source database, creating necessary users and permissions, and executing the migration through the RDS console. Post-migration, it is essential to verify data integrity, optimize performance by updating statistics and rebuilding indexes, and implement security best practices such as disabling the DMS user and enforcing SSL/TLS connections. Common troubleshooting errors during migration include permission issues, incorrect WAL levels, and connection refusals, each with specific resolutions. Finally, to avoid ongoing charges, it is recommended to clean up resources created during the migration process, including deleting the DMS migration task, logical replication publications, Secrets Manager secrets, and the target RDS/Aurora instance.

Tech Optimizer

July 22, 2026

Migrating mission-critical payments at Nubank to Amazon Aurora PostgreSQL

Nubank, a digital banking platform with approximately 135 million customers in Brazil, Mexico, and Colombia, faced challenges with its payment infrastructure due to inefficiencies in managing 7.5 TB of self-hosted PostgreSQL databases. Queries took over 13 minutes to execute, and replication lags exceeded four minutes during peak times. To address these issues, Nubank evaluated database solutions based on developer productivity, operational efficiency, performance reliability, and scalability. They selected Amazon Aurora PostgreSQL-Compatible Edition for its compatibility, performance improvements, and automated features. The migration to Aurora was facilitated by AWS Database Migration Service (AWS DMS), which minimized downtime and allowed for efficient transition. Post-migration, query performance improved significantly, with some queries executing up to 1,900 times faster, and overall end-to-end service latency decreased, enhancing customer experience. The migration resulted in a 25 percent cost reduction and ensured compliance with regulatory requirements.