database Archives

Tech Optimizer

July 24, 2026

How we pushed CDC into Postgres, and turned replication into clockwork

Making data from transactional databases accessible to analytical databases is essential in modern data architecture, but it faces challenges such as fragile tooling, high costs, and complex operations. Snowflake has developed a Postgres service that addresses these issues by reimagining Postgres replication. Postgres is a strong operational database, but its change data capture (CDC) capabilities need improvement. Many data pipelines are fragile due to complexities in managing continuous data flow, schema changes, snapshots, and failures. To enhance user experience with Snowflake Postgres, a complete reinvention of the replication process was necessary. A new feature called data mirroring is in public preview, allowing resilient data replication into Snowflake with low cost, minimal lag, and transactional consistency. This feature directly pushes changes from Postgres into Apache Iceberg™ tables in transactional batches, which are then automatically applied to Snowflake tables without extra infrastructure. Change data capture records changes from a transactional database for replay on another system, primarily using logical decoding in Postgres. This method decodes WAL records into logical operations, but the client bears the responsibility for subsequent steps like backfilling, managing schema changes, and handling failures. Built-in logical replication in Postgres only addresses some complexities. A limitation of logical decoding is that the external system does not recognize the state of Postgres, making it difficult to detect schema changes or correlate snapshots with changes. The solution is to push changes from Postgres into a data lake, specifically into Iceberg tables using compressed Parquet format, with object stores like Amazon S3 as the destination. The mirroring process uses a new Postgres extension called snowflake_cdc, which continuously pushes batches of changes into per-table change logs and a meta log. This extension understands the Postgres environment, coordinating schema changes and complex transactions while ensuring alignment between snapshots and changes.

Tech Optimizer

July 23, 2026

Optimizing PostgreSQL on Azure directly in Visual Studio Code

Poor database performance can lead to missed SLAs, delayed releases, customer dissatisfaction, and lost revenue. Microsoft has enhanced PostgreSQL on Azure, transforming it into a fully managed platform that meets enterprise demands. Azure Database for PostgreSQL and the new Azure HorizonDB offer significant performance improvements, with HorizonDB being three times faster than self-managed PostgreSQL. The PostgreSQL extension for Visual Studio Code integrates performance management into development workflows, providing tools for managing PostgreSQL throughout its lifecycle. Key features include a server metrics dashboard that displays performance indicators, Azure Advisor recommendations for actionable insights, improved query plan visualization, and AI-assisted query analysis. The extension also enhances schema design and query formulation experiences, ensuring secure and governed interactions with PostgreSQL. Azure HorizonDB is in public preview, designed for AI-native workloads. These advancements aim to reduce friction, enhance clarity, and enable faster actions for enterprise teams managing PostgreSQL at scale.

AppWizard

July 23, 2026

Active Exploitation Alert: Fake Bahrain Alert App Deploys Advanced Android Surveillance Malware Targeting Gulf Region Users

A cyber-espionage campaign has been identified involving a counterfeit Bahrain Alert Android application designed to surveil individuals in Bahrain and the Gulf region. The app masquerades as an official civil defense tool and employs social engineering tactics to compromise Android devices, extract sensitive data, and maintain remote access. It targets high-value individuals such as activists and journalists, particularly during civil unrest and missile alerts, leveraging trusted government branding to increase infection rates. The malware features a complex, multi-stage architecture with advanced evasion techniques and is distributed through phishing links and malicious websites. It initiates a four-stage infection process, ultimately installing a remote access trojan (RAT) that enables covert surveillance and encrypted communications. The malware can monitor device activity, capture credentials, and intercept communications, while also employing mechanisms to avoid detection and maintain persistence. The campaign primarily targets Bahraini citizens, exploiting fear during crises to encourage app installation. Although there are indications of Russian-speaking developers involved, there is no definitive attribution to a specific nation-state or APT group. Mitigation strategies include monitoring for suspicious app installations, user education on verifying app authenticity, and blocking known malicious domains.

Tech Optimizer

July 23, 2026

Simplify AI agent orchestration with Lakebase Postgres

The auditing process has traditionally been labor-intensive, requiring extensive document reviews. CLA (CliftonLarsonAllen LLP) partnered with the Databricks Forward Deployed Engineering team to develop a document processing application that reduces extraction time from hours to minutes while maintaining quality. The application is built on the Databricks platform, utilizing technologies like Lakebase Postgres, Databricks Apps, Lakeflow Jobs, MLflow, and Unity Catalog Volumes. The application addresses challenges in document parsing, including unpredictable per-task latency, rate-limit-aware throttling, workload prioritization, cost attribution per task, and real-time progress visibility. It eliminates the need for multiple specialized systems by integrating components such as a web application for user interaction, Lakebase for state management, an orchestrator for task management, and AI agents for document processing. The task queue is supported by two Postgres tables in Lakebase, tracking task status and execution attempts. Concurrency safety is ensured through a locking mechanism, and crash recovery is facilitated by lease-based locking. The orchestrator implements rate-limit-aware throttling and idempotent webhook callbacks to manage task processing effectively. An operator dashboard provides real-time metrics on task status, agent performance, and workload costs, with live updates through Postgres LISTEN/NOTIFY events. Cost attribution is managed at the application level, allowing for detailed monitoring of expenditures. Lakebase enhances the orchestration process with features like autoscaling compute, OAuth-rotated authentication, Unity Catalog integration, and support for branching and snapshots.

Tech Optimizer

July 22, 2026

Migrating mission-critical payments at Nubank to Amazon Aurora PostgreSQL

Nubank, a digital banking platform with approximately 135 million customers in Brazil, Mexico, and Colombia, faced challenges with its payment infrastructure due to inefficiencies in managing 7.5 TB of self-hosted PostgreSQL databases. Queries took over 13 minutes to execute, and replication lags exceeded four minutes during peak times. To address these issues, Nubank evaluated database solutions based on developer productivity, operational efficiency, performance reliability, and scalability. They selected Amazon Aurora PostgreSQL-Compatible Edition for its compatibility, performance improvements, and automated features. The migration to Aurora was facilitated by AWS Database Migration Service (AWS DMS), which minimized downtime and allowed for efficient transition. Post-migration, query performance improved significantly, with some queries executing up to 1,900 times faster, and overall end-to-end service latency decreased, enhancing customer experience. The migration resulted in a 25 percent cost reduction and ensured compliance with regulatory requirements.

Tech Optimizer

July 22, 2026

Google boosts AlloyDB vector search with HNSW preview

Google has introduced a preview of its columnar engine-accelerated HNSW feature for AlloyDB, enhancing vector search throughput by up to four times for users of the pgvector extension in PostgreSQL. This improvement is aimed at approximate nearest neighbor searches using HNSW across large datasets. AlloyDB utilizes a columnar engine that allows HNSW indexes to reside in memory, improving throughput and recall metrics. In benchmark tests on the GloVe 100 Angular dataset, queries per second increased by approximately 4.2x to 4.9x, with recall improving from about 0.78 to over 0.94 at a throughput of around 350 queries per second. The columnar engine's design bypasses traditional PostgreSQL buffer management bottlenecks, enhancing efficiency during graph traversal. The memory architecture is distinct from basic caching techniques, and the feature requires no changes to application code. Users must enable the columnar engine and index caching flags to utilize this feature. Benchmarks were conducted on an AlloyDB C4A machine with 16 virtual CPUs.

Tech Optimizer

July 21, 2026

I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer

The author transitioned from a data analyst role to a data engineer, creating a 12-month self-study roadmap focused on learning by building projects. The first project was a GitHub ETL pipeline, which evolved from a simple Python script to a more complex system using SQLite and GitHub Actions for automation. The author realized that the challenges of data engineering lay in system design and orchestration rather than just writing ETL logic. For the second project, the author built an automated RSS ingestion pipeline to explore engineering decisions in creating a reliable data pipeline. The project emphasized the importance of separating application responsibilities from orchestration tasks, leading to the use of Docker for consistent execution environments and Kestra for orchestration. The author validated each component of the project incrementally: first the Python ETL, then PostgreSQL, followed by Docker, and finally Kestra. This approach ensured that each layer functioned correctly before adding complexity. The final architecture included distinct responsibilities for each component: Python for data processing, PostgreSQL for data storage, Docker for execution, and Kestra for orchestration. The author learned that effective engineering involves considering system reliability, error handling, and observability, shifting the focus from merely writing code to designing entire systems. The experience transformed the author's mindset, emphasizing the importance of incremental building and the separation of execution and orchestration responsibilities.

Winsage

July 18, 2026

Windows 11 Secure Boot certificates have expired, and Microsoft says what to do next

Microsoft held its OEM Secure Boot Office Hours event on July 15, where engineers collaborated with OEM representatives from companies like Acer, Asus, Cisco, Dell, and HP. IT administrators were able to ask live questions about the Secure Boot 2023 rollout. The discussion thread became a detailed technical record, especially following the expiration of the first certificates three weeks prior. Concerns raised by IT admins included BitLocker recovery loops, stuck confidence ratings, and unhelpful Intune error codes. Key facts include: - Devices offline for long periods will still receive the 2023 certificates upon reconnecting to Windows Update. - Devices with existing 2023 certificates in firmware will switch to the new boot manager after the latest Windows patches are installed. - A new script, Detect-SecureBootCertUpdateStatus.ps1, is available in Windows for checking certificate status. - BIOS updates may reset a device’s confidence rating to unrated, which is normal and does not indicate certificate failure. - Admins should edit the AvailableUpdates registry key, not the AvailableUpdatesPolicy, which is managed by Intune and Group Policy. - A licensing bug affecting AvailableUpdatesPolicy on devices upgraded from Pro to Enterprise was resolved by Microsoft in 2026. - BitLocker recovery is not typically linked to the certificate update process but may relate to firmware or PCR issues. - Dell and HP provided guidance on which BIOS versions include the 2023 certificates for their newer models. - Older HP EliteBook 840 G5 units require a manual update package for the new certificates. - Eligible devices can still receive the 2023 certificates in the future, and Surface devices released from 2024 onward come pre-equipped with them. - Microsoft confirmed that devices running 2011 certificates will not lose the ability to receive the 2023 chain. - The Microsoft Corporation KEK CA 2011 and Microsoft UEFI CA 2011 certificates have expired, with the Microsoft Windows Production PCA 2011 set to expire on October 19, 2026.

Tech Optimizer

July 16, 2026

Google Cloud boosts AlloyDB search for CJK text with Gemini

Google Cloud has enhanced AlloyDB by integrating AlloyDB AI Functions with Gemini models to improve search indexing for logographic languages like Chinese, Japanese, and Korean. This update addresses challenges in database search functionality, particularly for languages written in continuous scripts, where traditional PostgreSQL text parsing struggles. Developers can now invoke Gemini models directly from SQL to facilitate word segmentation and stop-word removal before creating full-text indexes, offering a more efficient alternative to traditional methods. The architecture maintains raw text, segmented text, search vectors, and embeddings in a single table, with generated columns ensuring automatic updates to search vectors and embeddings. A batching process using a PL/pgSQL stored procedure allows for efficient processing of extensive document collections by aggregating rows into arrays for model requests, thus reducing row-locking issues. Google Cloud provides tailored text search configurations for different datasets, suggesting a simple configuration for solely Chinese text and an English configuration for bilingual datasets. This preprocessing logic can also be applied during query execution, allowing user searches to be directed to Gemini for text segmentation and stop-word elimination. AlloyDB supports RUM indexes on search vectors, which store lexeme positions for computing relevance and word distance, enabling rapid query responses. Additionally, a hybrid search methodology combines full-text and vector search using a ScaNN index on embeddings, merging results through Reciprocal Rank Fusion. This development allows for robust multilingual search and vector retrieval solutions within the database environment, eliminating the need for external microservices.

Winsage

July 15, 2026

LabubaRAT malware infiltrates Windows systems while posing as NVIDIA software

LabubaRAT is a newly identified Rust-based remote access tool (RAT) that disguises itself as NVIDIA software to facilitate operations on Windows systems. It establishes a reusable foothold for hands-on activity and can profile the host system, identify security tools, execute commands, transfer files, capture screenshots, and proxy network traffic. LabubaRAT retrieves its configuration at launch via command-line arguments or environment variables, allowing operators to specify parameters like the command-and-control server and API key at runtime. It maintains local state in a SQLite database and supports communication through HTTPS polling, Microsoft Edge WebView2, and DNS tunneling. The malware includes capabilities for command execution, file handling, and establishing persistence through a Windows Run registry key. The entry point for the attack is an executable named nvidia-sysruntime.exe, which masquerades as NVIDIA’s container runtime toolkit. LabubaRAT conducts thorough profiling of the host, checking for installed browsers and security products, and gathers information on the system's specifications. Blackpoint Cyber has shared indicators of compromise to help detect LabubaRAT activity.