PostgreSQL

Tech Optimizer
March 19, 2026
Postgres extensions, specifically pg_lake, pg_partman, and pg_incremental, provide a vendor-agnostic, open-source solution for managing high-performance time series data. PostgreSQL serves as the foundation, while pg_partman automates time partition management, pg_lake connects Postgres with data lakes for offloading cold data, and pg_incremental processes append-only data in batches. These extensions are maintained by the Postgres team at Snowflake. An example system for monitoring temperature readings uses local storage for recent data and transitions cold data to an Apache Iceberg table on S3. The process includes creating a partitioned table in Postgres, establishing an Iceberg table, using pg_incremental for data appending, eliminating old partitions with pg_partman, and querying from both local and cold tables to optimize storage and costs. A time-partitioned table enhances performance by allowing efficient deletion of outdated data, speeding up queries, and reducing fragmentation.
Tech Optimizer
March 18, 2026
Microsoft has launched a new database management tool called Database Hub, which is part of its Fabric data platform. This tool allows for the management of multiple databases using a single SQL engine and serves as a centralized location for overseeing various database services, including Azure SQL Server, Azure Cosmos DB, Azure Database for PostgreSQL, SQL Server with Azure Arc, and Azure Database for MySQL. Currently in early access, Database Hub aims to provide a unified management experience across on-premises, PaaS, and SaaS environments. The tool will incorporate AI capabilities to enhance database management, featuring "agent-assisted" and "human-in-the-loop" approaches for analyzing estate-wide signals. Microsoft’s LLM tool, Copilot, will offer insights into the database estate's status, including aggregate health views and trend analysis. However, specific details on the Database Hub's database tuning and optimization capabilities were not provided. Additionally, a study from Carnegie Mellon University indicated that vector embedding algorithms could improve the performance of PostgreSQL services significantly. Microsoft has also launched various database systems, including a document database platform based on PostgreSQL and a distributed PostgreSQL database service. Other vendors like Databricks and Snowflake have introduced their own transactional database services within their platforms.
Tech Optimizer
March 18, 2026
AWS has ended standard support for PostgreSQL 13 on its RDS platform, urging customers to upgrade to PostgreSQL 14 or later. PostgreSQL 14 introduces a new password authentication scheme (SCRAM-SHA-256) that disrupts the functionality of AWS Glue, which cannot accommodate this authentication method. Users upgrading to PostgreSQL 14 may encounter an error stating, "Authentication type 10 is not supported," affecting their data pipeline operations. The incompatibility has been known since PostgreSQL 14's release in 2021, and the deprecation timeline for PG13 was communicated in advance. AWS Glue's connection-testing infrastructure relies on an internal driver that predates the newer authentication support, leading to failures when validating setups. Customers face three options: downgrade to a less secure password encryption, use a custom JDBC driver that disables connection testing, or rewrite ETL workflows as Python shell jobs. Extended Support for customers who remained on PG13 is automatically enabled unless opted out during cluster creation, costing [openai_gpt model="gpt-4o-mini" prompt="Summarize the content and extract only the fact described in the text bellow. The summary shall NOT include a title, introduction and conclusion. Text: AWS PostgreSQL 13 Support Ends, Unveiling Compatibility Challenges Earlier this month, AWS concluded standard support for PostgreSQL 13 on its RDS platform, urging customers to upgrade to PostgreSQL 14 or later to maintain a supported database environment. This transition aligns with PostgreSQL 13's community end-of-life, which occurred late last year. PostgreSQL 14, introduced in 2021, enhances security by adopting a new password authentication scheme known as SCRAM-SHA-256. However, this upgrade inadvertently disrupts the functionality of AWS Glue, the managed ETL (extract-transform-load) service, which is unable to accommodate the new authentication method. Consequently, users who heed AWS's security recommendations may find themselves facing an error message stating, "Authentication type 10 is not supported," effectively halting their data pipeline operations. This situation is particularly concerning as both RDS and Glue are typically utilized within production environments, where reliability is paramount. The deprecation of PostgreSQL 13 did not create this issue; rather, it eliminated the option to bypass a long-standing problem that has persisted for five years. Customers now face a dilemma: either accept an increased maintenance burden or incur costs associated with Extended Support. The crux of the matter lies in the connection-testing infrastructure of AWS Glue, which relies on an internal driver that predates the newer authentication support. When users click the "Test Connection" button to validate their setup, it fails to function as intended. A community expert on AWS's support forum acknowledged three years ago that an upgrade to the driver was pending, assuring users that crawlers would operate correctly. However, reports have surfaced indicating that crawlers also encounter issues, further complicating the situation. This incompatibility has been acknowledged since PostgreSQL 14's release, and the deprecation timeline for PG13 was communicated in advance. Both the RDS and Glue teams are likely aware of industry developments, yet it appears that neither team monitored the implications of their respective updates on one another. The underlying reason for this disconnect is rooted in AWS's organizational structure, which comprises tens of thousands of engineers divided into numerous semi-autonomous service teams. Each team operates independently, with the RDS team focusing on lifecycle deprecations and the Glue team managing driver dependencies. Unfortunately, this division of responsibilities has resulted in a lack of ownership over the gap between the two services, leaving customers to confront the consequences in their production environments. This scenario is not indicative of malice or a deliberate revenue enhancement strategy; instead, it reflects the challenges posed by organizational complexity. Integration testing across service boundaries is inherently difficult, particularly when those boundaries span multiple billion-dollar businesses under the same corporate umbrella. The unfortunate outcome is that customers are left to grapple with the fallout of these misalignments. For those facing a broken pipeline in the early hours of the morning, the rationale behind the incompatibility becomes irrelevant. The pressing need is for a solution, and AWS has presented three options, none of which are particularly appealing: Downgrade the password encryption on your database to the older, less secure standard, which contradicts AWS's own security guidance. Utilize a custom JDBC driver, which disables connection testing and may not support all desired features. Reconstruct ETL workflows as Python shell jobs, effectively abandoning the benefits of a managed service. For customers who opted to remain on PG13 to avoid this specific issue, Extended Support is now automatically enabled unless explicitly opted out during cluster creation—a detail that can easily be overlooked. This support incurs a fee of [cyberseo_openai model="gpt-4o-mini" prompt="Rewrite a news story for a technical publication, in a calm style with creativity and flair based on text below, making sure it reads like human-written text in a natural way. The article shall NOT include a title, introduction and conclusion. The article shall NOT start from a title. Response language English. Generate HTML-formatted content using tag for a sub-heading. You can use only , , , , and HTML tags if necessary. Text: Earlier this month, AWS ended standard support for PostgreSQL 13 on RDS. Customers who want to stay on a supported database — as AWS is actively encouraging them to do — need to upgrade to PostgreSQL 14 or later. This makes sense, as PostgreSQL (pronounced POST-gruh-SQUEAL if, like me, you want to annoy the living hell out of everyone within earshot) 13 reached its community end of life late last year. PostgreSQL 14, which shipped in 2021, defaults to a more secure password authentication scheme (SCRAM-SHA-256, for any nerds that have read this far without diving for their keyboards to correct my previous parenthetical). It also just so happens to break AWS Glue, their managed ETL (extract-transform-load) service, which cannot handle that authentication scheme. If you upgrade your RDS database to follow AWS's own security guidance, AWS's own data pipeline tooling responds with "Authentication type 10 is not supported" and stops working. Given that both of these services tend to hang out in the environment that most companies call "production," this is not terrific! The deprecation didn't create this problem. It just removed the ability to avoid a problem that has existed for five years, unless you take on an additional maintenance burden or pay the Extended Support tax. Here's the technical shape of the Catch-22, stripped to what matters: when you move to a newer PostgreSQL on RDS, Glue's connection-testing infrastructure uses an internal driver that predates the newer authentication support. The "Test Connection" button — the thing you'd click to verify that your setup works before trusting it with production data — simply doesn't. A community expert on AWS's support forum acknowledged three years ago that "the tester is pending a driver upgrade," and assured users that crawlers use their own drivers and should work fine. Users in the same thread reported back that the crawlers also fail. Running Glue against RDS PostgreSQL is a bread-and-butter data engineering pattern, not an edge case — this is a well-paved path that AWS has let fall into disrepair. The incompatibility has been known since PostgreSQL 14 shipped in 2021. The deprecation timeline for PG13 was announced in advance. Both teams—RDS and Glue—presumably track industry developments. Neither, apparently, bothered to track each other. The charitable read on how this happens is also the correct one: AWS has tens of thousands of engineers organized into hundreds of semi-autonomous service teams. The RDS team ships deprecations on the RDS lifecycle, the Glue team maintains driver dependencies on the Glue roadmap, and nobody explicitly owns the gap between them. The customer discovers the incompatibility in production, usually at an inconvenient hour. This is not a conspiracy, as AWS lacks the internal cohesion needed to pull one of those off. This is also not a carefully-constructed revenue-enhancement mechanism, because the Extended Support revenue is almost certainly a rounding error on AWS's balance sheet compared to the customer ill-will it generates. Instead, this is simply organizational complexity doing what organizational complexity does. It's the same reason your company's internal tools don't talk to each other; AWS is just doing it at a scale where the blast radius is someone else's production database. Integration testing across service boundaries is genuinely hard when those boundaries span multiple billion-dollar businesses that happen to share a parent company. Nobody woke up and decided to break Glue. It came that way from the factory. I want to be clear that I genuinely believe this, because the alternative I'm about to describe isn't about intent. The problem with the charitable read is that it doesn't matter If you're staring at a broken pipeline in your environment at 2 am, the reason is academic. You need a fix. AWS has provided three of them, and they all suck. You can downgrade password encryption on your database to the older, less secure standard: the one you just upgraded away from, per AWS's own recommendations. You can bring your own JDBC driver, which disables connection testing and may not support all the features you want. Or you can rewrite your ETL workflows as Python shell jobs. Every exit means giving up the entire value proposition of a managed service — presumably why you're in this mess to begin with — or walking back the security improvement you were just told to make. For customers who stayed on PG13 to avoid this specific problem, Extended Support is now running automatically unless you opted out at cluster creation time—a detail that's easy to miss. That's $0.10 per vCPU-hour for the first two years, doubling in year three. A 16-vCPU Multi-AZ instance works out to nearly $30,000 per year in Extended Support fees alone. It's not a shakedown. But it is a number that appears on a bill, from a company that also controls the timeline for fixing the problem, and all of the customer response options are bad. AWS doesn't need to be running a shakedown. They just need to be large enough that the result is indistinguishable from one. This pattern isn't unique to AWS, and it isn't going away. Every major cloud provider – indeed, every major technology provider – is a portfolio of semi-autonomous teams whose roadmaps occasionally collide in their customers' environments. It will happen again, with different services and different authentication protocols and different billing line items. The question isn't whether the org chart will produce another gap like this. It will. The question is what happens after the gap appears: does the response look like accountability — acknowledging the incompatibility before the deprecation deadline, not after — or does it look like a shrug and three paid alternatives? Never attribute to malice what can be adequately explained by one very large org chart. Just don't forget to check the invoice. ®" temperature="0.3" top_p="1.0" best_of="1" presence_penalty="0.1" ].10 per vCPU-hour for the first two years, doubling in the third year. For instance, a 16-vCPU Multi-AZ instance could result in nearly ,000 annually in Extended Support fees alone. While this may not be a deliberate exploitation of customers, it does present a significant financial burden, especially given that AWS controls the timeline for resolving the underlying problem. This pattern of organizational dissonance is not unique to AWS; it is a common occurrence among major cloud providers and technology companies alike. Each operates as a collection of semi-autonomous teams, leading to potential conflicts that can manifest in customer environments. The future will likely see similar gaps arise, characterized by different services, authentication protocols, and billing implications. The critical question remains: how will these organizations respond once such gaps are identified? Will they demonstrate accountability by acknowledging incompatibilities before deprecation deadlines, or will they offer a shrug accompanied by three costly alternatives? In navigating this complex landscape, it is essential to remember that the challenges posed by large organizational structures can often lead to unintended consequences. As customers, vigilance regarding invoices and service compatibility is paramount." max_tokens="3500" temperature="0.3" top_p="1.0" best_of="1" presence_penalty="0.1" frequency_penalty="frequency_penalty"].10 per vCPU-hour for the first two years and doubling in the third year. This situation reflects the challenges posed by AWS's organizational complexity, where independent teams may not effectively coordinate updates, leading to customer difficulties.
Tech Optimizer
March 17, 2026
Microsoft is enhancing PostgreSQL to establish it as a high-performance, scalable, and enterprise-ready open database platform, addressing the limitations of legacy systems like Oracle. Many Oracle customers face rising licensing costs, performance bottlenecks, and scalability issues, prompting them to consider migration. Apollo Hospitals successfully migrated from Oracle to Azure Database for PostgreSQL, achieving a 60% reduction in operational costs and a threefold improvement in system performance. Microsoft has introduced an AI-assisted migration tool to simplify the transition from Oracle to PostgreSQL, automating the conversion of schemas and application code. Azure Database for PostgreSQL offers high performance, scalability, and security, with features like v6-series compute SKUs and SSD v2 storage. Azure HorizonDB, a new cloud-native PostgreSQL service, supports extreme performance demands and is designed for real-time analytics. Microsoft is committed to enhancing PostgreSQL as an open-source database for enterprise workloads, enabling organizations to innovate and become more agile.
Tech Optimizer
March 17, 2026
EnterpriseDB (EDB) has advanced its integration with NVIDIA's cuDF for Apache Spark to enhance Postgres® performance on NVIDIA AI infrastructure, achieving analytics capabilities up to 100 times faster than traditional methods. EDB emphasizes the need for a real-time analytics framework to address challenges posed by fragmented data silos and inefficient analytics processes. Key features of EDB Postgres AI include GPU-acceleration for interactive analytics, NVIDIA NIM model serving, fully air-gapped support for private registries, and high-speed retrieval with NVIDIA NeMo Retriever. Research indicates only 13% of enterprises have transitioned to production-scale agentic deployments, which report five times higher ROI. The EDB PG AI Analytics Engine can achieve 50–100x faster analytics on large datasets, supports lakehouse architectures, and ensures workload isolation and governance. EDB PG AI is positioned as a secure, compliant, and scalable platform for operationalizing data and AI workloads.
Tech Optimizer
March 13, 2026
EnterpriseDB (EDB) has launched the Postgres Vitality Index, which ranks commercial contributors to PostgreSQL, highlighting EDB's claim of over 30% of contributions. The index evaluates contributions based on core code, ecosystem extensions, and community support. EDB is positioned ahead of AWS and Microsoft in contributions. EDB's product strategy includes EDB Postgres AI, addressing data sovereignty and governance for AI systems, emphasizing a hybrid architecture for various workloads. EDB aims to enhance Postgres's readiness for enterprise AI, while also publishing resources related to Postgres and AI architecture. The index's introduction aligns with an increase in managed Postgres offerings and tools from various vendors.
Tech Optimizer
March 12, 2026
EnterpriseDB (EDB) has launched the Postgres Vitality Index to evaluate the commercial contributions shaping PostgreSQL's future. The index indicates that Postgres is the most strategically invested-in database globally, with EDB as the leading contributor, accounting for over 30% of contributions. Postgres is favored by over 55% of developers, reflecting its growing adoption among major tech firms. The index assesses contributions in three areas: core advancements in the PostgreSQL codebase, ecosystem enhancements through extensions and tools, and community support. EDB's Postgres AI platform addresses the need for data and AI sovereignty, offering a hybrid architecture for governance and flexibility. EDB has also published a guide on building AI platforms with Postgres, which will be distributed at NVIDIA's GTC event.
Tech Optimizer
March 11, 2026
By 2024, 78% of organizations are expected to utilize AI, a significant increase from previous years. However, 90% of technology leaders struggle to measure the return on investment from AI initiatives. Traditional databases are inadequate for AI applications due to limitations in features like vector similarity search and semantic retrieval. Many organizations face challenges in integrating AI applications with existing databases, particularly when migrating legacy systems to the cloud. Security and compliance are critical for AI applications in regulated industries, requiring audit trails, data encryption, and adherence to standards like HIPAA and GDPR. The absence of dedicated vendors for transitioning AI from prototyping to production is notable, with no Postgres vendor focusing solely on AI integration until recently. Anthropic's open-source Model Context Protocol (MCP) has emerged as a standard for connecting AI agents to data sources, easing integration challenges. The underlying database architecture is crucial for supporting enterprise-grade AI applications, with Postgres being a common choice. The pgEdge Agentic AI Toolkit for Postgres provides a solution for building production-ready AI applications while ensuring availability, security, and compliance.
Tech Optimizer
March 11, 2026
Netflix has developed an internal automation platform to migrate Amazon RDS for PostgreSQL databases to Amazon Aurora PostgreSQL, reducing operational risks and downtime for nearly 400 production clusters. The platform allows service teams to perform migrations through a self-service workflow while ensuring processes like replication validation and rollback safeguards are maintained. Database access is managed through a platform-managed layer using Envoy, which standardizes mutual TLS and abstracts database endpoints, enhancing security and efficiency. The migration process starts with creating an Aurora PostgreSQL cluster as a read replica of the source RDS instance, initialized from a storage snapshot and continuously replaying write-ahead log (WAL) records. Validation checks are performed to ensure the replica can handle peak write throughput before cutover. For change data capture workloads, the system coordinates the state of replication slots and pauses CDC consumers to prevent excessive WAL retention. The Enablement Applications team at Netflix successfully migrated databases for device certification and partner billing workflows, addressing issues like elevated replication lag due to inactive logical replication slots. As replication lag decreases, the system enters a controlled quiescence phase, adjusts security rules, and reboots the source RDS instance. Once all transactions are processed and the Aurora replica is ready, it is promoted to a writable cluster, and traffic is rerouted. Rollback capabilities are prioritized, allowing redirection back to the original RDS instance if validation checks fail or anomalies are detected post-promotion. This setup enables seamless restoration without redeployment, and CDC consumers can resume from recorded slot positions if needed.
Search