NewApp Digest
search bot

Aurora PostgreSQL

Migrating mission-critical payments at Nubank to Amazon Aurora PostgreSQL
Tech Optimizer
July 22, 2026

Migrating mission-critical payments at Nubank to Amazon Aurora PostgreSQL

Nubank, a digital banking platform with approximately 135 million customers in Brazil, Mexico, and Colombia, faced challenges with its payment infrastructure due to inefficiencies in managing 7.5 TB of self-hosted PostgreSQL databases. Queries took over 13 minutes to execute, and replication lags exceeded four minutes during peak times. To address these issues, Nubank evaluated database solutions based on developer productivity, operational efficiency, performance reliability, and scalability. They selected Amazon Aurora PostgreSQL-Compatible Edition for its compatibility, performance improvements, and automated features. The migration to Aurora was facilitated by AWS Database Migration Service (AWS DMS), which minimized downtime and allowed for efficient transition. Post-migration, query performance improved significantly, with some queries executing up to 1,900 times faster, and overall end-to-end service latency decreased, enhancing customer experience. The migration resulted in a 25 percent cost reduction and ensured compliance with regulatory requirements.
Running pgvector in production on Amazon Aurora PostgreSQL
Tech Optimizer
July 12, 2026

Running pgvector in production on Amazon Aurora PostgreSQL

Running pgvector on Amazon Aurora PostgreSQL-Compatible Edition offers a vector store with operational capabilities, high availability, and scalability. It is favored for Retrieval Augmented Generation (RAG) workloads transitioning to production, but increased traffic introduces challenges like query latency and memory management. Key operational practices for pgvector workloads include selecting the appropriate index type (HNSW or IVFFlat), establishing a baseline schema, choosing a suitable distance operator, scaling the index through quantization and partitioning, and preparing for churn and observability. The prerequisites for using pgvector include an Aurora PostgreSQL-Compatible cluster with specific PostgreSQL versions and the vector extension enabled. The embedding model used in examples is Amazon Titan Text Embeddings V2, which produces 1024-dimensional embeddings. pgvector supports two Approximate Nearest Neighbor (ANN) index types: HNSW, which is efficient for querying and allows for incremental insertions, and IVFFlat, which is less resource-intensive but requires rebuilding if data changes. There are scenarios where forgoing an index is beneficial, such as small datasets or partitioned datasets requiring 100% recall. A baseline schema for a multi-tenant document store includes creating a table for documents with an embedding vector and establishing indexes for tenant IDs and embeddings using HNSW. The recommended parameters for HNSW include m = 16 and ef_construction = 128. Scaling to millions of vectors involves quantization, tuning HNSW parameters, and partitioning. Aurora Optimized Reads can extend effective cache capacity, and managing index churn is crucial for maintaining performance. Observability metrics include query-level statistics, instance-level metrics, and custom application-defined metrics. To clean up after testing, it is advisable to drop the created indexes and tables, and delete the Aurora PostgreSQL-Compatible cluster and any manual snapshots taken during testing.
Logical replication improvements in Amazon RDS for PostgreSQL 18
Tech Optimizer
July 8, 2026

Logical replication improvements in Amazon RDS for PostgreSQL 18

PostgreSQL logical replication has operational challenges, such as the absence of generated columns in the replication stream, replication conflicts causing apply worker halts, and lingering replication slots consuming write-ahead log (WAL) files. PostgreSQL 18 introduces enhancements to address these issues, including the ability to include STORED generated columns in the replication stream with a new publication parameter, detailed conflict counters in pg_stat_subscription_stats, automatic invalidation of idle replication slots, and the ability to toggle two-phase commit settings on live subscriptions. To set up a test environment, two PostgreSQL 18 instances (one publisher and one subscriber) are required, along with an EC2 instance for executing commands. The publisher and subscriber must have specific configurations for logical replication, including enabling logical replication and setting appropriate parameters. PostgreSQL 18 allows the replication of STORED generated columns by using the publish_generated_columns parameter. The subscriber must define these columns as regular columns. The new version also enhances conflict monitoring by providing detailed counters for different types of replication conflicts. Parallel streaming is now the default behavior for subscriptions, improving efficiency. The two-phase commit feature can now be modified on existing subscriptions without needing to recreate them. PostgreSQL 18 also introduces automatic cleanup of idle replication slots, which can be configured to invalidate unused slots after a specified duration. Considerations for implementation include ensuring that the subscriber's column types match those of the publisher for generated columns, the requirement for track_commit_timestamp for certain conflict counters, and the fact that sequences are not replicated. Cleanup of replication resources is recommended after testing to prevent unnecessary costs and WAL accumulation.
PostgreSQL 18 on Amazon Aurora and Amazon RDS: Security, monitoring, and developer enhancements
Tech Optimizer
June 20, 2026

PostgreSQL 18 on Amazon Aurora and Amazon RDS: Security, monitoring, and developer enhancements

PostgreSQL version 18 has deprecated MD5 password authentication in favor of SCRAM-SHA-256, with a new parameter, md5_password_warnings, enabled by default to log deprecation warnings. It has enhanced monitoring capabilities by adding columns to pg_stat_database and pg_stat_statements to track parallel worker activity, with the default max_parallel_workers_per_gather set to 0 in Aurora PostgreSQL. The pg_stat_subscription_stats view now includes new columns for tracking conflict types in logical replication. Optimizer statistics are automatically transferred during upgrades, while uuidv7() generates timestamp-ordered UUIDs. The default streaming option for CREATE SUBSCRIPTION has changed to parallel, and the idle_replication_slot_timeout parameter automatically invalidates inactive replication slots. Enhancements to the COPY command include REJECT_LIMIT for error tolerance and a silent LOG_VERBOSITY level. OLD and NEW aliases have been introduced in RETURNING clauses for various DML commands.
PostgreSQL 18 on Amazon Aurora and Amazon RDS: Performance enhancements
Tech Optimizer
June 20, 2026

PostgreSQL 18 on Amazon Aurora and Amazon RDS: Performance enhancements

PostgreSQL 18 addresses common performance challenges for users, including managing query performance across composite indexes, diagnosing memory spills in materialized Common Table Expressions (CTEs), and upgrading major versions without plan regressions. Key enhancements include skip scan optimization for multicolumn indexes, improved EXPLAIN functionality, and optimizer statistics that persist through major version upgrades. Skip scan optimization allows PostgreSQL to efficiently utilize multicolumn B-tree indexes even when leading columns are not specified in the WHERE clause, significantly improving query performance. The EXPLAIN command has been enhanced to include buffer statistics by default, providing deeper insights into query execution and resource usage. PostgreSQL 18 also introduces visibility into the storage of materialized nodes in query plans, indicating whether intermediate results were stored in memory or spilled to disk. A new metric, Index Searches, has been added to EXPLAIN ANALYZE output, indicating how many times the database traversed the index tree during query execution. Additionally, Self-Join Elimination (SJE) automatically detects and removes unnecessary inner joins of a table to itself, optimizing query performance. The autovacuum mechanism has been improved with the introduction of autovacuum_vacuum_max_threshold, which caps the number of dead tuples that can accumulate before autovacuum triggers a VACUUM, addressing issues with large tables. The vacuum_truncate parameter provides a server-wide control point to disable VACUUM’s file truncation behavior, reducing locking issues on busy systems. PostgreSQL 18 also separates the allocation of autovacuum worker slots from their usage, allowing for dynamic adjustments to autovacuum_max_workers without requiring a server restart. Finally, new columns in pg_stat_all_tables track cumulative time spent on maintenance operations, providing better insights into maintenance overhead for each table.
Azure HorizonDB Enters Public Preview: Web IQ Already Powers Copilot and ChatGPT
Tech Optimizer
June 6, 2026

Azure HorizonDB Enters Public Preview: Web IQ Already Powers Copilot and ChatGPT

Microsoft announced the public preview of Azure HorizonDB, a fully managed PostgreSQL-compatible database designed for agentic AI workloads, during Microsoft Build 2026 in San Francisco. HorizonDB features a "database-as-logs" architecture, allowing for sub-millisecond multi-zone commit latency and independent scaling of compute and storage. It incorporates a Rust-based storage engine, native DiskANN vector search, and in-database AI model invocation. Additionally, Microsoft launched Web IQ, a web-grounding API layer integrated into Microsoft Copilot and OpenAI's ChatGPT, which provides passage-level structured evidence objects rather than full documents. Web IQ is model-agnostic and aims to enhance information density and reduce costs. Both services are currently in limited availability, with HorizonDB open for preview signups across five Azure regions.
Search