Skip to main content

Descriptor classes and vector indexes

A descriptor stores its vector in the binary _blob column of a descriptor class. Each vector index on that column uses a single engine. Set kind to flat, ivf, hnsw, or acorn and put the settings directly in params. To use two engines on the same class, create an index of each kind. Each index can store application metadata in its own metadata object.

CreateIndex registers the class if necessary and populates the index with any existing vectors. You can also ingest vectors first with AddDescriptor. GetIndexes lists index definitions and their indexing progress.

Configuration and eligibility​

Every vector index specifies its dimensions and one or more distance metrics. All vector indexes on the same class must use the same dimensions. Insertions require a nonempty, finite float32 vector, including when a class has no index. Indexed vectors must match the declared dimensions.

L2 and IP include stored vectors whose squared magnitude fits the float range. Cosine excludes zero vectors and normalizes other eligible vectors. Finite vectors that are ineligible for a metric remain stored. Indexing progress describes the vectors eligible for each metric. Computing a nonfinite distance raises an error.

Engine settings​

The Flat table lists the parameters shared by all four kinds. IVF, HNSW, and ACORN also accept the settings in their respective tables. A nearest-neighbor query selects an index with the engine field. Its values are Flat, IVF, HNSW, and ACORN. Metadata-only descriptor queries can omit engine.

Flat parameters​

kind: "flat" performs exhaustive search and returns exact neighbors.

Parameter

Default

Meaning

dimensions

Required

Number of float32 values per vector, an integer from 1 through 32768.

dtype

"float32"

Vector element type. The supported value is "float32".

metric

"L2"

A metric name or a nonempty list of names. L2 measures squared Euclidean distance, IP measures inner product, and CS measures cosine similarity.

lazy_indexing

false

Set to true to process index changes in the background. Pending changes remain visible to searches.

IVF parameters​

kind: "ivf" groups vectors into clusters. A trained index searches the clusters closest to the query vector.

Parameter

Default

Meaning

nlist

100

Number of clusters, an integer from 1 through 2147483647. Zero selects the default.

nprobe

ceil(sqrt(nlist))

Number of clusters to search, an integer from 1 through the resolved nlist. Zero selects the default, which is 10 for 100 clusters.

IVF performs exact search until it has trained its centroids. Ordinary insertion triggers training at 39 * nlist live, eligible descriptors (3900 with the default settings). Synchronous indexing trains in the write transaction that reaches the threshold. Lazy indexing trains in the background. A complete synchronous rebuild stages all eligible descriptors before checking the threshold.

To train earlier, call UpdateIndex with build_index: true. The command applies pending changes and trains each untrained metric with at least nlist live, eligible descriptors. Already trained indexes keep their centroids. Searches never initiate training.

Training uses ten iterations, seed 1234, and spherical centroids for inner product. Ordinary insertion uses the first 39 * nlist live descriptors. A complete synchronous rebuild or explicit build uses the available live population, subject to a cap of 256 samples per list and the transaction workspace limit. Training uses a deterministic reservoir sample if needed and requires at least one sample per list. Changing nprobe alone updates the query budget without rebuilding the index.

HNSW parameters​

kind: "hnsw" builds a hierarchical navigable small world graph for approximate search.

Parameter

Default

Meaning

M

32

Graph degree, an integer from 2 through 1073741823. Cumulative neighbor offsets must also fit the native integer range. Zero selects the default.

efConstruction

40

Construction search budget, an integer from 1 through 2147483647. Zero selects the default.

efSearch

16

Query search budget, an integer from 1 through 2147483647. Zero selects the default. Increasing it can improve recall at the cost of query latency.

Construction also requires sufficient transaction workspace and record capacity.

ACORN parameters​

kind: "acorn" builds a graph with expanded upper-level links and compressed bottom-level links for filtered approximate search. ACORN automatically uses filtered traversal for constrained queries. Unconstrained queries use the same graph.

Parameter

Default

Meaning

M

32

Base graph degree, an integer from 2 through 4096.

gamma

4

Construction expansion factor, an integer from 1 through 64. M * gamma must fit within 4096.

M_beta

M

Number of bottom-level neighbors retained before compression, an integer from 0 through 2 * M * gamma. The upper endpoint disables bottom-level compression.

efConstruction

200

Construction search budget, an integer from M through 16777216. The effective budget is at least 2 * M * gamma.

efSearch

32

Query search budget, an integer from 1 through 16777216. Increasing it can improve recall at the cost of query latency.

This example creates a cosine index for 128-dimensional vectors and increases the ACORN query budget to 64.

[{
"CreateIndex": {
"target": "descriptor",
"class": "filtered_embeddings",
"property": "_blob",
"kind": "acorn",
"params": {
"dimensions": 128,
"metric": "CS",
"efSearch": 64
}
}
}]

ACORN evaluates the matching subset exactly when fewer than 1 / gamma of the indexed vectors match, or when the subset contains at most 2 * M * max(k_neighbors, efSearch) vectors. Other filtered searches are approximate and can return fewer than k_neighbors results. Changing only efSearch updates the query budget without rebuilding the graph.

Search and lifecycle​

with_class restricts the participating class before top-k, including with knn_first: true. Omitting it searches across classes. Every participating populated class needs an index of the requested kind with matching dimensions and a common metric. An omitted metric selects the first common metric in L2/IP/CS order.

Each input vector gets one global top-k across participating classes. Exact search resolves ties by full descriptor identity before truncation. Filters run before top-k by default. knn_first: true applies them afterward. Result pagination applies after global selection.

Lazy indexing​

By default, descriptor writes and index changes commit in the same graph transaction. With params.lazy_indexing set to true, a write records the pending index changes in that transaction. Searches combine the stored index with pending additions, updates, and removals. Changes are immediately visible to later commands in the transaction and to new queries after commit. Pending changes can increase search latency until indexing catches up.

Both modes use the same bounded cache. A cache miss reads the stored index from the current snapshot while an optional cache copy is prepared in the background. Eligible batches of HNSW appends can use the cache for construction.

UpdateIndex replaces a selected index’s parameters. Disabling lazy indexing applies pending changes before committing. build_index: true finishes pending work and can train IVF while retaining the current generation and indexing mode. A change to construction settings can hold the writer while rebuilding. Metadata edits and class renames preserve index contents. Removing an index hides it immediately and releases its storage in the background. Recreating it allocates a fresh identity.