workers Archives

Tech Optimizer

July 24, 2026

How we pushed CDC into Postgres, and turned replication into clockwork

Making data from transactional databases accessible to analytical databases is essential in modern data architecture, but it faces challenges such as fragile tooling, high costs, and complex operations. Snowflake has developed a Postgres service that addresses these issues by reimagining Postgres replication. Postgres is a strong operational database, but its change data capture (CDC) capabilities need improvement. Many data pipelines are fragile due to complexities in managing continuous data flow, schema changes, snapshots, and failures. To enhance user experience with Snowflake Postgres, a complete reinvention of the replication process was necessary. A new feature called data mirroring is in public preview, allowing resilient data replication into Snowflake with low cost, minimal lag, and transactional consistency. This feature directly pushes changes from Postgres into Apache Iceberg™ tables in transactional batches, which are then automatically applied to Snowflake tables without extra infrastructure. Change data capture records changes from a transactional database for replay on another system, primarily using logical decoding in Postgres. This method decodes WAL records into logical operations, but the client bears the responsibility for subsequent steps like backfilling, managing schema changes, and handling failures. Built-in logical replication in Postgres only addresses some complexities. A limitation of logical decoding is that the external system does not recognize the state of Postgres, making it difficult to detect schema changes or correlate snapshots with changes. The solution is to push changes from Postgres into a data lake, specifically into Iceberg tables using compressed Parquet format, with object stores like Amazon S3 as the destination. The mirroring process uses a new Postgres extension called snowflake_cdc, which continuously pushes batches of changes into per-table change logs and a meta log. This extension understands the Postgres environment, coordinating schema changes and complex transactions while ensuring alignment between snapshots and changes.

Tech Optimizer

July 23, 2026

Simplify AI agent orchestration with Lakebase Postgres

The auditing process has traditionally been labor-intensive, requiring extensive document reviews. CLA (CliftonLarsonAllen LLP) partnered with the Databricks Forward Deployed Engineering team to develop a document processing application that reduces extraction time from hours to minutes while maintaining quality. The application is built on the Databricks platform, utilizing technologies like Lakebase Postgres, Databricks Apps, Lakeflow Jobs, MLflow, and Unity Catalog Volumes. The application addresses challenges in document parsing, including unpredictable per-task latency, rate-limit-aware throttling, workload prioritization, cost attribution per task, and real-time progress visibility. It eliminates the need for multiple specialized systems by integrating components such as a web application for user interaction, Lakebase for state management, an orchestrator for task management, and AI agents for document processing. The task queue is supported by two Postgres tables in Lakebase, tracking task status and execution attempts. Concurrency safety is ensured through a locking mechanism, and crash recovery is facilitated by lease-based locking. The orchestrator implements rate-limit-aware throttling and idempotent webhook callbacks to manage task processing effectively. An operator dashboard provides real-time metrics on task status, agent performance, and workload costs, with live updates through Postgres LISTEN/NOTIFY events. Cost attribution is managed at the application level, allowing for detailed monitoring of expenditures. Lakebase enhances the orchestration process with features like autoscaling compute, OAuth-rotated authentication, Unity Catalog integration, and support for branching and snapshots.

AppWizard

July 16, 2026

Today’s Android app deals and freebies: MEGA MAN X, Little Big Workshop, Ace Attorney Trilogy, more

A selection of Android games and apps are available at discounts, including MEGA MAN X for (reg. ), Little Big Workshop for .25 (reg. ), Ace Attorney Trilogy for (reg. ), GHOST TRICK for (reg. ), LEGO Bricktales for (reg. ), and Neighbours from Hell 1 Premium for (reg. ). The Google Pixel Flex Dual Port 67W USB-C Fast Charger is now priced at , reflecting a 25% discount. The Google Nest Doorbell is available for (reg. 0). The most affordable unlocked Pixel 10 is up to 8 off, while Pro models are reduced by up to 4. The Google Pixel Watch 4 is currently 2 off.

AppWizard

July 15, 2026

Mid-week Android app deals and freebies: ScourgeBringer, Retro Zoo CEO, Final Fantasy Dimensions, more

Today's Android game and app deals include titles such as ScourgeBringer, Retro Zoo CEO, Guardian War: Ultimate Edition, Final Fantasy Dimensions, Tiny Terraces, and Beat Workers. The official Google Pixel Flex Dual Port 67W USB-C Fast Charger is discounted by 25%. The Pixel 10 Pro is available at a discount of 0, and the Pixel Watch 4 is offered for less than its standard price. Google Pixel 10/Pro Pixelsnap cases have discounts of up to 36%. There are significant price reductions on several Android apps, reflecting a trend of accessibility and affordability in the mobile app sector. Additional Android app price drops continue to be available throughout the day.

Winsage

July 11, 2026

I tested Acer’s new Swift Go 16 AI, and this affordable OLED laptop impressed me more than I had expected

The Acer Swift Go 16 AI features an Intel Core Ultra 7 355 processor, 32GB LPDDR5X RAM, integrated Intel Graphics, a 16" WUXGA (1920 x 1200) non-touch OLED display at 60Hz, a 1TB M.2 NVMe PCIe 4.0 SSD, a 71WHrs battery with a 100W USB-C charger, and weighs 2.95 lbs (1.34 kg). It has a sleek design with a reflective Acer logo and geometric patterns. Performance tests indicate it outperforms some competitors in multi-core tasks and offers quick boot times and efficient multitasking. The display provides vibrant colors but maxes out at 300 nits. It includes a backlit keyboard, a mechanical trackpad, a webcam with Windows Hello and Human Presence Detection, and has a battery life of 8 to 14 hours. Drawbacks include slight chassis flex, subpar audio quality, and pre-installed bloatware.

Winsage

July 10, 2026

MS Office Pro Plus 2019 and Windows 11 Pro are now just $26

The All-in-One Microsoft Office Pro 2019 for Windows: Lifetime License + Windows 11 Pro Bundle is priced at .99, reduced from its regular price of 9.99. It includes Microsoft Office Professional Plus 2019, which offers offline access to applications like Word, Excel, PowerPoint, Outlook, OneNote, Publisher, and Access, along with updates for improved functionality. The bundle also features Windows 11 Pro, which includes tools for multitasking and organization, as well as advanced options for business network management and device security.

AppWizard

July 9, 2026

MindsEye developers protest against a fan event their own studio is hosting

MindsEye, an action game from Build a Rocket Boy, is set to have a playtest at the studio's Edinburgh headquarters, inviting community members and covering their travel costs. This comes after significant layoffs that affected over 400 employees, leading to protests organized by the Independent Workers Union of Great Britain, which criticizes the studio for spending on the playtest while having recently laid off staff. The union also alleges the studio has engaged in questionable practices, including employee surveillance. The playtest is seen as a strategy to regain favor with fans amid concerns about the studio's management and treatment of its workforce.

AppWizard

July 8, 2026

Mid-week Android app deals and freebies: Beat Workers, Tiny Terraces, Potion Permit, more

A selection of Android games is available at discounted prices, including Beat Workers, Tiny Terraces, Potion Permit, SENTINEL 519: SAGA – GENESIS, Frontline: Western Front, Burning Sky VIP, and Raiders of the North Sea. The Galaxy Z Fold 7 models are discounted by up to 0, the Pixel Watch 4 has a price cut of nearly 0, and the Sonos Ace ANC headphones are available for 9, reflecting a discount. Additional app price drops and exclusive offers are also available for Android users.

Tech Optimizer

July 8, 2026

PostgreSQL Extensions Are an Architecture Decision, Not an Afterthought

A common issue in database migrations is the unplanned accumulation of extensions in PostgreSQL, leading to extension sprawl. Teams often install extensions without documenting the rationale, resulting in a complex web of dependencies that complicates future upgrades and removals. The installation process involves PostgreSQL accessing a control file that details the extension's version and dependencies, which can lead to multiple extensions being installed unintentionally. Upgrading and removing extensions are often neglected, causing risks such as the loss of dependent objects. Extensions typically default to the public schema, which can become cluttered; relocating them to dedicated schemas can improve organization. Trusted Language Extensions (TLE) allow non-privileged users to utilize procedural languages in managed databases without needing superuser access. Key extensions recommended for use include pg_stat_statements, pg_trgm, hstore, citext, and PostGIS, each serving specific use cases. Proper governance is essential for managing extensions, including documenting their purpose, ownership, and dependencies, to prevent operational surprises.

Tech Optimizer

July 8, 2026

Logical replication improvements in Amazon RDS for PostgreSQL 18

PostgreSQL logical replication has operational challenges, such as the absence of generated columns in the replication stream, replication conflicts causing apply worker halts, and lingering replication slots consuming write-ahead log (WAL) files. PostgreSQL 18 introduces enhancements to address these issues, including the ability to include STORED generated columns in the replication stream with a new publication parameter, detailed conflict counters in pg_stat_subscription_stats, automatic invalidation of idle replication slots, and the ability to toggle two-phase commit settings on live subscriptions. To set up a test environment, two PostgreSQL 18 instances (one publisher and one subscriber) are required, along with an EC2 instance for executing commands. The publisher and subscriber must have specific configurations for logical replication, including enabling logical replication and setting appropriate parameters. PostgreSQL 18 allows the replication of STORED generated columns by using the publish_generated_columns parameter. The subscriber must define these columns as regular columns. The new version also enhances conflict monitoring by providing detailed counters for different types of replication conflicts. Parallel streaming is now the default behavior for subscriptions, improving efficiency. The two-phase commit feature can now be modified on existing subscriptions without needing to recreate them. PostgreSQL 18 also introduces automatic cleanup of idle replication slots, which can be configured to invalidate unused slots after a specified duration. Considerations for implementation include ensuring that the subscriber's column types match those of the publisher for generated columns, the requirement for track_commit_timestamp for certain conflict counters, and the fact that sequences are not replicated. Cleanup of replication resources is recommended after testing to prevent unnecessary costs and WAL accumulation.