Descriptor classes and vector indexes
A descriptor stores its vector in the binary _blob column of a descriptor
class. Each vector index on that column uses a single engine. Set kind to
flat, ivf, hnsw, or acorn and put the settings directly in
params. To use two engines on the same class, create an index of each kind.
Each index can store application metadata in its own metadata object.
CreateIndex registers the class if necessary and populates the index with any existing vectors. You can also ingest vectors first with AddDescriptor. GetIndexes lists index definitions and their indexing progress.
Configuration and eligibility
Every vector index specifies its dimensions and one or more distance metrics. All vector indexes on the same class must use the same dimensions. Insertions require a nonempty, finite float32 vector, including when a class has no index. Indexed vectors must match the declared dimensions.
L2 and IP include stored vectors whose squared magnitude fits the float range. Cosine excludes zero vectors and normalizes other eligible vectors. Finite vectors that are ineligible for a metric remain stored. Indexing progress describes the vectors eligible for each metric. Computing a nonfinite distance raises an error.
Engine settings
The Flat table lists the parameters shared by all four kinds. IVF, HNSW, and
ACORN also accept the settings in their respective tables. A nearest-neighbor
query selects an index with the engine field. Its values are Flat, IVF,
HNSW, and ACORN. Metadata-only descriptor queries can omit engine.
Flat parameters
kind: "flat" performs exhaustive search and returns exact neighbors.
Parameter |
Default |
Meaning |
|---|---|---|
|
Required |
Number of float32 values per vector, an integer from 1 through 32768. |
|
|
Vector element type. The supported value is |
|
|
A metric name or a nonempty list of names. |
|
|
Set to |
IVF parameters
kind: "ivf" groups vectors into clusters. A trained index searches the
clusters closest to the query vector.
Parameter |
Default |
Meaning |
|---|---|---|
|
100 |
Number of clusters, an integer from 1 through 2147483647. Zero selects the default. |
|
|
Number of clusters to search, an integer from 1 through the resolved |
IVF performs exact search until it has trained its centroids. Ordinary
insertion triggers training at 39 * nlist live, eligible descriptors
(3900 with the default settings). Synchronous indexing trains in the write
transaction that reaches the threshold. Lazy indexing trains in the background.
A complete synchronous rebuild stages all eligible descriptors before checking
the threshold.
To train earlier, call UpdateIndex with build_index: true.
The command applies pending changes and trains each untrained metric with at
least nlist live, eligible descriptors. Already trained indexes keep their
centroids. Searches never initiate training.
Training uses ten iterations, seed 1234, and spherical centroids for inner
product. Ordinary insertion uses the first 39 * nlist live descriptors.
A complete synchronous rebuild or explicit build uses the available live
population, subject to a cap of 256 samples per list and the transaction
workspace limit. Training uses a deterministic reservoir sample if needed and
requires at least one sample per list. Changing nprobe alone updates the
query budget without rebuilding the index.
HNSW parameters
kind: "hnsw" builds a hierarchical navigable small world graph for
approximate search.
Parameter |
Default |
Meaning |
|---|---|---|
|
32 |
Graph degree, an integer from 2 through 1073741823. Cumulative neighbor offsets must also fit the native integer range. Zero selects the default. |
|
40 |
Construction search budget, an integer from 1 through 2147483647. Zero selects the default. |
|
16 |
Query search budget, an integer from 1 through 2147483647. Zero selects the default. Increasing it can improve recall at the cost of query latency. |
Construction also requires sufficient transaction workspace and record capacity.
ACORN parameters
kind: "acorn" builds a graph with expanded upper-level links and compressed
bottom-level links for filtered approximate search. ACORN automatically uses
filtered traversal for constrained queries. Unconstrained queries use the same
graph.
Parameter |
Default |
Meaning |
|---|---|---|
|
32 |
Base graph degree, an integer from 2 through 4096. |
|
4 |
Construction expansion factor, an integer from 1 through 64. |
|
|
Number of bottom-level neighbors retained before compression, an integer from 0 through |
|
200 |
Construction search budget, an integer from |
|
32 |
Query search budget, an integer from 1 through 16777216. Increasing it can improve recall at the cost of query latency. |
This example creates a cosine index for 128-dimensional vectors and increases the ACORN query budget to 64.
[{
"CreateIndex": {
"target": "descriptor",
"class": "filtered_embeddings",
"property": "_blob",
"kind": "acorn",
"params": {
"dimensions": 128,
"metric": "CS",
"efSearch": 64
}
}
}]
ACORN evaluates the matching subset exactly when fewer than 1 / gamma of
the indexed vectors match, or when the subset contains at most
2 * M * max(k_neighbors, efSearch) vectors. Other filtered searches are
approximate and can return fewer than k_neighbors results. Changing only
efSearch updates the query budget without rebuilding the graph.
Search and lifecycle
with_class restricts the participating class before top-k, including
with knn_first: true. Omitting it searches across classes. Every
participating populated class needs an index of the requested kind with
matching dimensions and a common metric. An omitted metric selects the first
common metric in L2/IP/CS order.
Each input vector gets one global top-k across participating classes. Exact
search resolves ties by full descriptor identity before truncation. Filters
run before top-k by default. knn_first: true applies them afterward. Result
pagination applies after global selection.
Lazy indexing
By default, descriptor writes and index changes commit in the same graph
transaction. With params.lazy_indexing set to true, a write records the
pending index changes in that transaction. Searches combine the stored index
with pending additions, updates, and removals. Changes are immediately visible
to later commands in the transaction and to new queries after commit. Pending
changes can increase search latency until indexing catches up.
Both modes use the same bounded cache. A cache miss reads the stored index from the current snapshot while an optional cache copy is prepared in the background. Eligible batches of HNSW appends can use the cache for construction.
UpdateIndex replaces a selected index’s parameters. Disabling
lazy indexing applies pending changes before committing. build_index: true
finishes pending work and can train IVF while retaining the current generation
and indexing mode. A change to construction settings can hold the writer while
rebuilding. Metadata edits and class renames preserve index contents. Removing
an index hides it immediately and releases its storage in the background.
Recreating it allocates a fresh identity.