NewApp Digest
search bot

scripts

I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer
Tech Optimizer
July 21, 2026

I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer

The author transitioned from a data analyst role to a data engineer, creating a 12-month self-study roadmap focused on learning by building projects. The first project was a GitHub ETL pipeline, which evolved from a simple Python script to a more complex system using SQLite and GitHub Actions for automation. The author realized that the challenges of data engineering lay in system design and orchestration rather than just writing ETL logic. For the second project, the author built an automated RSS ingestion pipeline to explore engineering decisions in creating a reliable data pipeline. The project emphasized the importance of separating application responsibilities from orchestration tasks, leading to the use of Docker for consistent execution environments and Kestra for orchestration. The author validated each component of the project incrementally: first the Python ETL, then PostgreSQL, followed by Docker, and finally Kestra. This approach ensured that each layer functioned correctly before adding complexity. The final architecture included distinct responsibilities for each component: Python for data processing, PostgreSQL for data storage, Docker for execution, and Kestra for orchestration. The author learned that effective engineering involves considering system reliability, error handling, and observability, shifting the focus from merely writing code to designing entire systems. The experience transformed the author's mindset, emphasizing the importance of incremental building and the separation of execution and orchestration responsibilities.
Windows 11 Secure Boot certificates have expired, and Microsoft says what to do next
Winsage
July 18, 2026

Windows 11 Secure Boot certificates have expired, and Microsoft says what to do next

Microsoft held its OEM Secure Boot Office Hours event on July 15, where engineers collaborated with OEM representatives from companies like Acer, Asus, Cisco, Dell, and HP. IT administrators were able to ask live questions about the Secure Boot 2023 rollout. The discussion thread became a detailed technical record, especially following the expiration of the first certificates three weeks prior. Concerns raised by IT admins included BitLocker recovery loops, stuck confidence ratings, and unhelpful Intune error codes. Key facts include: - Devices offline for long periods will still receive the 2023 certificates upon reconnecting to Windows Update. - Devices with existing 2023 certificates in firmware will switch to the new boot manager after the latest Windows patches are installed. - A new script, Detect-SecureBootCertUpdateStatus.ps1, is available in Windows for checking certificate status. - BIOS updates may reset a device’s confidence rating to unrated, which is normal and does not indicate certificate failure. - Admins should edit the AvailableUpdates registry key, not the AvailableUpdatesPolicy, which is managed by Intune and Group Policy. - A licensing bug affecting AvailableUpdatesPolicy on devices upgraded from Pro to Enterprise was resolved by Microsoft in 2026. - BitLocker recovery is not typically linked to the certificate update process but may relate to firmware or PCR issues. - Dell and HP provided guidance on which BIOS versions include the 2023 certificates for their newer models. - Older HP EliteBook 840 G5 units require a manual update package for the new certificates. - Eligible devices can still receive the 2023 certificates in the future, and Surface devices released from 2024 onward come pre-equipped with them. - Microsoft confirmed that devices running 2011 certificates will not lose the ability to receive the 2023 chain. - The Microsoft Corporation KEK CA 2011 and Microsoft UEFI CA 2011 certificates have expired, with the Microsoft Windows Production PCA 2011 set to expire on October 19, 2026.
Explore a snapshot of a massive million-by-million block Minecraft server
AppWizard
July 17, 2026

Explore a snapshot of a massive million-by-million block Minecraft server

The 1,024,000² 2b2t World Download Project is a significant Minecraft archiving achievement, preserving approximately 13.7 TiB of compressed world data from the 2b2t server, including a million-by-million-block section of the Overworld, and smaller captures of the Nether and End dimensions. It is the largest available Minecraft world download, released via torrent. The project took over a year to develop, involving a team that utilized 28 bot accounts to collect data, resulting in four distinct captures of varying sizes. The project was led by a developer named crayne, with key contributors including Fuch, mahan, and Steve3. The code and documentation are available on GitHub, and the project incurred over ,000 in server rental costs. 2b2t, established in December 2010, is one of the oldest Minecraft servers, known for its unyielding map and challenging environment, having hosted over 1,000,000 players. The creators assert that no AI was used in the project and have licensed their work under CC0 for community accessibility.
PC Gamer Repurposes Old SSDs to Create a Steam Game Cartridge System
AppWizard
July 17, 2026

PC Gamer Repurposes Old SSDs to Create a Steam Game Cartridge System

Jibril-sama, a Redditor, has transformed old 2.5" SSDs into a new gaming cartridge system, combining nostalgia with modern technology. The SSDs, acquired for approximately each, serve as the basis for this project, which involves minor software tweaks and custom printed labels. The setup allows games stored on the SSDs to auto-navigate to their Steam pages and can auto-start games. This approach offers a practical solution for PC gamers facing rising storage costs, utilizing spare drives while maintaining SATA III speeds. The configuration requires a systemd template and a udev rule to launch scripts from the SSDs.
Google Cloud boosts AlloyDB search for CJK text with Gemini
Tech Optimizer
July 16, 2026

Google Cloud boosts AlloyDB search for CJK text with Gemini

Google Cloud has enhanced AlloyDB by integrating AlloyDB AI Functions with Gemini models to improve search indexing for logographic languages like Chinese, Japanese, and Korean. This update addresses challenges in database search functionality, particularly for languages written in continuous scripts, where traditional PostgreSQL text parsing struggles. Developers can now invoke Gemini models directly from SQL to facilitate word segmentation and stop-word removal before creating full-text indexes, offering a more efficient alternative to traditional methods. The architecture maintains raw text, segmented text, search vectors, and embeddings in a single table, with generated columns ensuring automatic updates to search vectors and embeddings. A batching process using a PL/pgSQL stored procedure allows for efficient processing of extensive document collections by aggregating rows into arrays for model requests, thus reducing row-locking issues. Google Cloud provides tailored text search configurations for different datasets, suggesting a simple configuration for solely Chinese text and an English configuration for bilingual datasets. This preprocessing logic can also be applied during query execution, allowing user searches to be directed to Gemini for text segmentation and stop-word elimination. AlloyDB supports RUM indexes on search vectors, which store lexeme positions for computing relevance and word distance, enabling rapid query responses. Additionally, a hybrid search methodology combines full-text and vector search using a ScaNN index on embeddings, merging results through Reciprocal Rank Fusion. This development allows for robust multilingual search and vector retrieval solutions within the database environment, eliminating the need for external microservices.
PC gamer turns Steam games into cartridges with ingenious 2.5-inch SSD system — games are stored on 128GB drives alongside a script to auto-start the title once plugged in
AppWizard
July 15, 2026

PC gamer turns Steam games into cartridges with ingenious 2.5-inch SSD system — games are stored on 128GB drives alongside a script to auto-start the title once plugged in

A Reddit user named Jibril-sama has created a Steam Game Cartridge system that uses old SATA SSDs as game cartridges, allowing users to access their Steam library in a nostalgic format. The system includes a disk mount detection and execution script for easy game launching, alleviating storage issues for gamers. It operates on Linux, utilizing Valve’s Steam URL Protocol for navigation. The setup requires a systemd template and a udev rule to function properly. There are concerns about the compatibility of Steam game libraries when transferred across different PCs, and there is potential for Jibril-sama to share guides and resources for others to replicate the project.
2026's best RPG might look like Disco Elysium, but it was heavily inspired by Deus Ex, and its creator wants 'to do a first-person game at some point'
AppWizard
July 12, 2026

2026’s best RPG might look like Disco Elysium, but it was heavily inspired by Deus Ex, and its creator wants ‘to do a first-person game at some point’

Christoffer Bodegård began his journey into PC gaming with a DOS machine from his father and first played Disney's The Jungle Book. His passion for role-playing games was ignited by Dragon Age: Origins, which led him to explore many titles, including Planescape: Torment. After eight years of development, his game Esoteric Ebb launched in March to critical acclaim. Currently, he is playing Victoria 3, having logged around 400 hours, and previously played Abiotic Factor. The oldest games on his PC are Heroes of Might and Magic 3, King of Dragon Pass, Deus Ex, and Diablo 2, with Heroes 3 being a favorite. He has logged around 1,000 hours in Esoteric Ebb and is particularly fond of Europa Universalis IV, especially the Anbennar mod. Bodegård has never uninstalled Deus Ex, which greatly influences his design philosophy. He relies on Notepad++ for scripting and has improved his desktop organization from a cluttered state to a cleaner setup.
Search