Skip to content
← Ecosystem

Vector index recall

pg_recall_guard v0.2.10

A degraded ANN index does not fail: it returns plausible neighbours and never mentions the ones it left out.

It measures the recall of any vector index against exact ground truth, records the number you accepted as good, and tells you when it drifts. With no error, no log entry and no alert, you would find out when a user says the search "feels worse".

Install

CREATE EXTENSION pg_recall_guard;

In plain words

It measures whether your smart search still finds what it should, and warns when it starts failing without anyone noticing.

  1. The situation

    Your assistant looks for documents "similar" to what it is asked: the semantic search AI systems use.

  2. Without this piece

    Over time, the index behind that search degrades: it returns plausible results, but the best ones slip away. There is no error to see.

  3. With this piece

    Your team approves a number ("it finds 98% of what it should") and a scheduled measurement recomputes it. When it drops, monitoring says so: ok, degraded, critical.

Why it matters if you decide

  • Search quality as a number, not a feeling
  • Works with any vector index, today's and whatever gets installed tomorrow
  • Declared maturity: measured on one dataset, no production workload yet
For the technical team

The problem

Four of the ten good neighbours are missing and nobody says so

The degradation of an approximate index is gradual and silent: the only moment it gets caught is when something bothers to look.

It works with any vector index because it never names one: they are found through pg_amop (access methods that can answer an ORDER BY on a distance operator). Today that covers hnsw and ivfflat (pgvector), diskann (pgvectorscale), gist and spgist; tomorrow it covers whatever ships next, with no change here.

How it works

Approve a baseline, measure on a schedule, read a verdict

  1. 01

    vector_indexes

    What can be measured, discovered from the catalog with no configuration: index, table, access method, operator.

  2. 02

    approve(index, p_k, p_sample_size)

    Record what you consider good. Sample queries are drawn from rows of the table itself.

  3. 03

    check()

    One row per baseline with a verdict: ok (within 0.02 of the baseline), degraded (0.02 to 0.10 below), critical (more than 0.10), or COULD NOT MEASURE with the error (index dropped, the planner chose another one).

  4. 04

    Built so it cannot flatter itself

    The indexed side must use THIS index (if the planner picks another, the function says which); the ground truth must not use it; k+1 neighbours are asked for and the originating ctid discarded (a vector always finds itself: with k=10 recall could never drop below 0.1); ties are judged by distance; a row is its table and its ctid (partitions).

SQL

Usage

Every function runs with search_path = pg_catalog, pg_temp and a baseline names its index with the schema: check() measures the same index from your session, from pg_cron, or from a session whose path lacks that schema; a temporary table cannot take its place (test/pg_temp.sh, red on 0.2.3 and green from 0.2.4).

psql · pg_recall_guardsql
CREATE EXTENSION pg_recall_guard;

-- What can be measured? Discovered from the catalog.
SELECT index_name, table_name, access_method, operator
  FROM recall_guard.vector_indexes;

-- Record what you consider good.
SELECT recall_guard.approve('items_hnsw', p_k => 10, p_sample_size => 100);

-- Later, from pg_cron or your monitoring:
SELECT * FROM recall_guard.check();
--     index_name     | baseline | current |  drift  | verdict
-- -------------------+----------+---------+---------+---------
--  public.items_hnsw |   0.9800 |  0.8100 | -0.1700 | critical

What it guarantees

What it guarantees

  • A reassuring 1.0000 that means nothing is ruled out by construction: the index measured is the one approved and the ground truth does not use it.
  • Measuring an index runs nothing its table's owner wrote (0.2.6): the sampled vector is cast to the base type, not to a domain with its CHECKs.
  • Matrix 10 through 19 green with make installcheck in official containers (measured 2026-09-16).
  • It measures the index under the settings it was approved with: approve() records the search settings (hnsw.*, ivfflat.*, diskann.*) and check() measures under them, so no role can make a degraded index pass with a SET.
  • It is neutral: it belongs to no index, so it can watch all of them.

What it does not do

Maturity, said before you rely on it

  • check() runs one exact sequential scan per sampled query: not free and not meant for every request. It is for a schedule, on a sample, the way you would run ANALYZE. Start with p_sample_size => 30.
  • The performance figures come from one machine and one dataset (38,352 vectors, 768 dimensions) on hnsw and diskann. Other dimensions and parameters are untested.
  • Nobody has run this against a production workload yet; 0.1.0 was testing and nobody could find it. Treat the recall numbers as trustworthy and the coverage as narrow.
  • The sampling strategy (vectors from the table itself) is a reasonable proxy for real traffic, not real traffic: capturing actual queries needs an executor hook in C, which this does not have.
  • Since 0.2.9 verdicts are in English (they were Spanish): a monitor filtering on "degradado" or "critico" must be updated.
  • check() measures the index, not the application's configuration: if someone lowers ivfflat.probes for the whole application, this does not see it.
  • A baseline goes by name: an index dropped and created again with the same name inherits it. Re-approve after such a change.

For the enterprise

For the enterprise

The measurement missing from every semantic search system in production: an approved number, a schedule that re-measures it, and a verdict your monitoring understands.

  • Index-independent: works with pgvector today and with whatever gets installed tomorrow
  • Verdicts with fixed thresholds (0.02 / 0.10) that do not move on their own
  • Reports from other datasets are the most useful thing anyone can send the project

This piece is not in the Enterprise 0.3.1 image, which ships pg_agent_gate, pg_living_assertions, pg_promise_guard and pg_grammar_guard. Install it separately from PGXN.

Spec sheet

Version
0.2.10 on PGXN; not in the public image 0.3.6 (measured 2026-11-13)
PostgreSQL
10 through 19 green (installcheck, 2026-09-16); performance measured on 18.6 and 19beta2
Depends on
tsm_system_rows (contrib); works with pgvector, pgvectorscale and any index with distance ordering operators
License
Apache-2.0
Enterprise edition
Not in the Enterprise 0.3.1 image
Try it Talk