We value your privacy

We use cookies to enhance your browsing experience, serve personalized ads or content, and analyze our traffic. By clicking "Accept All", you consent to our use of cookies.

Customize Consent Preferences

We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.

The cookies that are categorized as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... 

Always Active

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

No cookies to display.

Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

No cookies to display.

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

No cookies to display.

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

No cookies to display.

Advertisement cookies are used to provide visitors with customized advertisements based on the pages you visited previously and to analyze the effectiveness of the ad campaigns.

No cookies to display.

NewApp Digest
search bot

indexing

Supercharging vector search performance and relevance with pgvector 0.8.0 on Amazon Aurora PostgreSQL
Tech Optimizer
June 1, 2025

Supercharging vector search performance and relevance with pgvector 0.8.0 on Amazon Aurora PostgreSQL

The Amazon Aurora PostgreSQL-Compatible Edition has integrated support for pgvector 0.8.0, enhancing vector search capabilities significantly. This version achieves up to 9x faster query processing and delivers 100x more relevant search results compared to previous iterations. Key improvements in pgvector 0.8.0 include: - Up to 5.7x improvement in query performance for specific patterns. - Introduction of the iterative_scan feature, enhancing recall for filter queries requiring approximate nearest neighbor (ANN) index searches. - Improved cost estimation for more efficient query planning, allowing the selection of traditional indexes for complex filtered searches. - Flexible performance tuning with two modes for iterative_scan: relaxed_order and strict_order. Previous versions faced challenges with overfiltering, where combining vector similarity search with traditional SQL filters resulted in fewer results than expected. The HNSW indexing algorithm accelerates vector similarity searches by creating a multi-layered graph structure for efficient navigation. Iterative index scans in pgvector 0.8.0 improve query reliability and performance by applying filters during the scanning process, reducing false negatives and enhancing performance. The configuration options for iterative scanning include off, strict_order, and relaxed_order, with relaxed_order providing the best balance between performance and accuracy. In performance tests, pgvector 0.8.0 demonstrated substantial improvements across various query patterns, with significant enhancements in result completeness and recall. The database can now efficiently handle enterprise-scale applications, such as e-commerce and recommendation systems, with high performance and result quality even as product catalogs expand. Best practices for deploying pgvector 0.8.0 include using HNSW with recommended parameters, creating additional indexes on commonly filtered metadata columns, and tuning query parameters based on specific use cases. Operational best practices involve using Graviton4-based instances for optimal performance, monitoring and tuning slow vector queries, and considering partitioning for very large tables.
Signal resorts to “weird trick” to block Windows Recall in Desktop app
AppWizard
May 22, 2025

Signal resorts to “weird trick” to block Windows Recall in Desktop app

Microsoft's Recall feature indexes a wide range of personal data, including Zoom meetings, emails, photos, medical conditions, and conversations on Signal, affecting both users and their contacts without consent. Researcher Kevin Beaumont found that the feature captures sensitive information like payment card details and can decrypt its database using a fingerprint scan or PIN. Developers, such as those at Signal, lack tools to prevent their content from being indexed by Recall, leading Signal to utilize a Digital Rights Management API to protect privacy. This workaround may help, but it depends on all chat participants using the Windows Desktop version with default settings. Microsoft has not addressed concerns regarding developer control over Recall.
How to catch and optimize problematic queries in PostgreSQL
Tech Optimizer
May 19, 2025

How to catch and optimize problematic queries in PostgreSQL

Performance issues in PostgreSQL can arise as databases grow, leading to declines in query performance. To identify problematic queries, one can log long-running queries by setting the `log_min_duration_statement` parameter in the `postgresql.conf` file. The `pg_stat_statements` module can be enabled to monitor execution statistics of SQL statements, providing insights into query performance, including execution time and the number of calls. Queries with high standard deviation in execution time may indicate inconsistency, while sorting query statistics by total execution time can reveal excessive load from multiple fast queries. Real-time monitoring can be done using the `pg_stat_activity` view to check active queries and their states, and the `pg_locks` view can help identify blocked processes. The `EXPLAIN` command can analyze query execution plans, and using the `auto_explain` module can log plans for slow queries. Understanding cost parameters in query plans helps in optimizing performance, and different join methods (Nested Loop, Merge Join, Hash Join) have varying complexities and costs. To influence the planner's choice of scanning methods, configuration parameters can be adjusted, such as disabling sequential scanning. Extensions like `sr_plan`, `pg_hint_plan`, and `AQO` can further optimize query execution. Monitoring query progress can be done using dynamic views like `pg_stat_progress_*` for various commands.
The Search Stack That Nearly Broke Us — Until We Did This
Tech Optimizer
May 8, 2025

The Search Stack That Nearly Broke Us — Until We Did This

The initial architecture for the finlight.me real-time financial news API utilized Postgres full-text search, which was effective for early needs but faced limitations as article volume increased. Key challenges included performance degradation due to the inability to combine GIN indexes with regular B-Tree indexes, leading to slow query performance, especially with complex user queries and pagination. The decision was made to transition to OpenSearch for its optimized search capabilities while retaining Postgres as the source of truth for data integrity. Testing with OpenSearch revealed the need for production-grade resources, which led to a successful deployment that improved search response times significantly. The current architecture separates ingestion, storage, and retrieval responsibilities, with Postgres managing data storage and OpenSearch handling user-facing search queries. Regular snapshotting and thoughtful index management ensure resilience and scalability.
Search