data duplication Archives

Tech Optimizer

March 29, 2026

Announcing Lakebase Public Preview

At the Data and AI Summit, Databricks announced the Public Preview of Databricks Lakebase, a fully-managed Postgres database designed for data applications and artificial intelligence. Lakebase addresses challenges in operational databases by merging operational and analytical data, offering a scalable architecture that separates compute from storage. It supports popular Postgres extensions and features low latency and high concurrency transactions. Lakebase includes multi-zone high availability, encrypted storage with point-in-time recovery, and copy-on-write branching for isolated testing. It integrates seamlessly with the Databricks lakehouse, enabling efficient data synchronization and unified governance. Lakebase is accessible across multiple cloud providers and is currently in Public Preview, with hundreds of customers already using it for various applications.

Tech Optimizer

March 6, 2026

Azure Databricks Lakebase is Generally Available

Azure Databricks Lakebase is a managed, serverless PostgreSQL solution optimized for the Databricks Platform on Azure, announced by Microsoft as generally available. It separates compute from storage, allowing direct writing of operational data to lakehouse storage and bridging the gap between transactional systems and analytics. Lakebase features instant branching and zero-copy clones, enhancing developer productivity by enabling safe testing environments without infrastructure delays. It operates on a serverless model with autoscaling capabilities, ensuring cost efficiency by charging users only for the compute resources utilized. Lakebase is built on standard PostgreSQL, ensuring compatibility with existing tools and libraries, and supports various extensions. It provides unified governance through Unity Catalog, offering consistent access control and auditing across the Azure Databricks data estate. The platform facilitates AI development by enabling real-time operational context access and low-latency feature serving. Azure Databricks Lakebase integrates with Microsoft Entra ID for security and compliance, simplifying the DevOps burden for developers.

Tech Optimizer

February 12, 2026

Snowflake unveils Postgres to make enterprise data AI-ready

Snowflake has introduced Snowflake Postgres, which will be generally available soon, designed to unify transactional workloads, analytics, and AI development within its AI Data Cloud. It is fully compatible with open-source Postgres, allowing for seamless migration of existing applications without code modifications. Snowflake Postgres integrates Apache Iceberg through pg_lake, enabling users to manage Iceberg tables using standard SQL, reducing data movement between systems. Companies like BlueCloud and Sigma Computing have adopted it for operational applications and real-time analytics. Alongside Snowflake Postgres, Snowflake has enhanced data governance and interoperability through the Snowflake Horizon Catalog, which allows for better access and governance across various systems. The Horizon Catalog supports querying Iceberg tables and managing data stored in them. Snowflake has also launched Open Format Data Sharing, extending its zero-ETL sharing model to open formats like Apache Iceberg and Delta Lake, and has integrated with Microsoft OneLake for secure data sharing. Additionally, Snowflake has made Snowflake Backups generally available to safeguard business-critical data and ensure compliance with regulatory requirements, allowing for quicker recovery from disruptions.

Tech Optimizer

February 12, 2026

Snowflake plugs PostgreSQL into its AI Data Cloud

Snowflake is launching a PostgreSQL database-as-a-service to enhance its AI Data Cloud, allowing organizations to integrate transactional workloads with analytics and AI under a unified governance framework. This service is fully compatible with open-source PostgreSQL, enabling easy migration of existing applications without code modifications. It utilizes pg_lake to read and write directly to Apache Iceberg tables, eliminating the need for data extraction and movement. Snowflake aims to reduce costly data movement between transactional and analytical systems, building on its previous transactional capability, Unistore. This strategic move positions Snowflake to offer a managed OLTP solution, facilitating the development of agentic AI and real-time streaming capabilities. The trend of combining operational databases with analytics is becoming common among vendors, with competitors like Databricks also launching similar services. By consolidating OLTP and OLAP capabilities, organizations can reduce ETL processes and data duplication while maintaining consistent governance across workloads.

Tech Optimizer

February 12, 2026

Databricks launches PostgreSQL Lakebase to aid AI developers

Databricks Lakebase has transitioned to general availability, launched on AWS on February 3, following the acquisition of Neon for billion in May 2025. Lakebase is a PostgreSQL database designed for AI development, integrating with Databricks' Data Intelligence Platform to provide an operational database alongside data lakehouse capabilities. It decouples compute from storage to improve resource management and includes autoscaling features to manage costs. Lakebase also offers unified governance through Databricks' Unity Catalog. Analysts highlight its ability to reduce friction between operational and analytical data, enabling real-time applications with up-to-date governed data and minimizing extensive ETL processes. Key features include serverless autoscaling and instant database branching for enhanced developer productivity. Databricks aims to simplify database management at scale and demonstrate a lower total cost of ownership to compete with Snowflake.

Tech Optimizer

February 12, 2026

pg_lake comes to Snowflake Postgres: What it means for open standards

The pg_lake extension has been integrated into Snowflake Postgres, enabling users to utilize PostgreSQL as a data lakehouse with support for Apache Iceberg tables. This integration allows for unified analytical and transactional workloads, facilitating direct access to PostgreSQL within the Snowflake environment. Snowflake has also expanded its Snowflake Horizon Catalog to enhance data discovery and governance, now managing data accessed through external query systems like Apache Spark. New features include integration with Microsoft One Lake for secure, bi-directional data access between Snowflake and Microsoft Fabric. pg_lake includes functionalities for version management, data type and format conversions, and caching to optimize performance. Snowflake's capabilities for managing Iceberg tables have been enhanced, allowing external query engines to read and write to these tables while extending access controls for data governance. Additionally, Snowflake's open data sharing features now include Delta Lake and Iceberg tables, enabling users to access data without duplication or movement across clouds. The expansion of open standards supports interoperability, simplifying developers' work with various tools and formats.

Tech Optimizer

January 27, 2026

PostgreSQL for WMS: a DBMS selection strategy in the era of import substitution

Choosing a Database Management System (DBMS) for Warehouse Management Systems (WMS) is a strategic decision that impacts security, budget, and adaptability. PostgreSQL is identified as a safe, cost-effective, and future-proof solution for Russian warehouse systems, especially following the withdrawal of foreign vendors like Oracle and Microsoft SQL Server. Starting September 1, 2025, foreign software usage in government bodies and companies with over 50% state ownership will face legal restrictions, posing regulatory risks for those using Oracle. Financial risks include high licensing fees and the potential for legal prohibitions on current technologies. Business continuity risks arise from the possibility of vendors ceasing support and updates. PostgreSQL, both in its vanilla form and commercial derivatives like Postgres Pro, is presented as a robust alternative capable of handling high loads and providing necessary features for modern WMS. It supports real-time analytics and advanced capabilities such as intelligent search and automatic classification using neural networks. Key considerations when choosing a DBMS include estimating data volumes, checking legal requirements, and calculating total cost of ownership (TCO).

Tech Optimizer

November 25, 2025

Protect sensitive data with dynamic data masking for Amazon Aurora PostgreSQL

Amazon Aurora PostgreSQL-Compatible Edition has introduced a dynamic data masking feature that provides column-level protection, enhancing security alongside PostgreSQL’s row-level security. This feature is available for Aurora PostgreSQL versions 16.10 and higher, as well as 17.6. Dynamic data masking allows organizations to maintain a single copy of their data while applying role-based access controls, ensuring sensitive information is protected in real-time without data duplication. The pg_columnmask extension enables policy-based masking and runtime query rewriting, allowing different users to see varying levels of data based on their roles. For example, customer service representatives may see only the last four digits of account numbers, while executives see full details. The feature supports compliance with regulations such as GDPR, HIPAA, and PCI DSS. Users can create masking policies using standard SQL commands, and the policies can be defined to utilize built-in or custom functions for data masking.

Winsage

October 8, 2025

How to deploy Data Duplication on Windows Server

Storage capacity is crucial for managing data costs and performance, leading to the use of data deduplication to reduce redundant data and optimize storage space. Microsoft’s Data Deduplication feature, introduced with Windows Server 2012, can achieve storage savings of up to 50% for user documents and 95% for virtualization libraries. Data deduplication is applicable to file servers, backup storage servers, and virtualization hosts, requiring Windows Server 2012 or later and the NTFS file system. To install Data Deduplication, users must access Server Manager, add the feature, and can also use PowerShell with the cmdlet PLACEHOLDERb80efd5ce6cbf150. Configuration involves managing settings through Server Manager, including selecting a deduplication type, setting a file age, and scheduling the deduplication process. Monitoring and optimizing deduplication can be done using the Data Deduplication Saves Evaluation Tool (ddpeval.exe) and PowerShell cmdlets like PLACEHOLDER6242a4d48a44de3e. Alternative deduplication products include Veeam Backup and Replication, Arcserve UDP, and Acronis Cyber Protect, which may offer additional features for cloud or hybrid environments. Best practices for deduplication include using the latest Windows Server versions, avoiding system volumes, ensuring adequate free space, and scheduling tasks during off-peak hours. Troubleshooting tips involve checking memory and processor performance, utilizing ddpeval.exe, and reviewing Event Viewer logs.

AppWizard

October 4, 2025

Helldivers 2 Dev Explains Why The Game Takes Up So Much Space On PC

Helldivers 2 has a storage requirement of 150GB on PC, which is about three times larger than on Xbox Series X|S and PlayStation 5. The large install size is designed to accommodate PC users with mechanical hard disk drives (HDDs) by duplicating data to improve loading times, although this is unnecessary for users with solid state drives (SSDs). Current estimates suggest that around 12% of PC gamers still use HDDs, but reliable data on this is lacking. Arrowhead Game Studio plans to reduce the game's file size in an upcoming update and bundle common assets to minimize duplication, though this may slightly increase load times for HDD users. The studio aims to keep load times under 30 seconds and is also working on improving its engine and compression techniques. Additionally, Arrowhead's CEO expressed a commitment to the game's longevity rather than developing a sequel, and the studio is addressing various technical issues with planned updates.