CREATE INDEX ... USING HNSW. VECTOR_SEARCH is a table‑valued function that returns up to top_k rows from the indexed table that are closest to a target vector according to the distance metric configured on the index.
Read more about Vector Search Indexes in general and how to create vector indexes.
Syntax
Parameters
EF_SEARCH
Theef_search parameter defines the breadth of exploration during query‑time search.
- Higher values improve recall; latency increases approximately with
ef_search. - Set
ef_search >= top_k. While this is not strictly required, it is generally recommended to achieve good recall. - Typical range:
50–200; recommended starting range:64–100. - Tune per workload; you can override per query without rebuilding the index.
Return type
A rowset with the same columns as the indexed base table. Returns up totop_k rows.
The rowset also has a $approx_distance column of type DOUBLE PRECISION: the approximate distance between the row’s quantized embedding and the target vector, measured with the index’s distance metric.
Like other pseudo-columns, $approx_distance is not part of SELECT *; name it explicitly to read it.
The rows are not returned in any particular order, so use ORDER BY $approx_distance to sort them.
Smaller values mean closer rows for every distance metric. For
vector_l2sq_ops, the value is the squared Euclidean distance; use SQRT($approx_distance) to get the result of VECTOR_EUCLIDEAN_DISTANCE.
For vector_ip_ops, the value ranks rows but is not a true distance: it is negative whenever the inner product exceeds 1, so it has no lower bound. For unit-length embeddings it equals the cosine distance. Use 1 - $approx_distance to get the inner product, where larger values mean closer rows.
The index computes the distance on vectors stored with its quantization, so $approx_distance can differ slightly from the expression in the table. Rows from tablets without an index get the exact value of that expression. If the table has its own column named $approx_distance, that column is returned instead.
WHERE condition on $approx_distance filters the top_k rows after the search, so the result can have fewer than top_k rows.
Notes
- When using
load_strategy => 'in_memory'(the default), you might want to adjust your engine configuraton to allow the in-memory vector index cache to use more memory usingVECTOR_INDEX_CACHE_MEMORY_FRACTION. Otherwise, vector indexes need to be loaded into memory in every query leading to slow perfomance. See Performance & observability for more information. - You must create and populate a vector search index before using
VECTOR_SEARCH. VECTOR_SEARCHuses the distance metric defined on the index (for example,vector_cosine_ops).- If the index was created on a populated table without reindexing, Firebolt combines results from indexed tablets with full scans of unindexed tablets. For best performance, run
VACUUM ( REINDEX = TRUE )on the table after creating the index. - To tune recall at query time, increase
ef_search.
Filtering
AWHERE condition on PARTITION BY or PRIMARY INDEX columns narrows the search to the data that can match. With a condition on a partition column, VECTOR_SEARCH returns up to top_k rows from the matching partitions:
top_k rows.
Examples
Create a table with embeddings, create a vector search index, then runVECTOR_SEARCH() .