External Search Index¶

Libraries such as FAISS implement various other indexing algorithms that offer much more flexibility than the baseline of pyvisim. To allow such indexes to be used for image similarity search in this library, the ExternalSearchIndex class is provided.

Note

Like the case of hnsw index, each index created can potentially increase memory usage, since each index has its own internal structure, built from the embeddings themselves (which, in the worst case, can be the same as the original embeddings).

Example¶

Tip

See this tutorial for a more detailed walkthrough.

import faiss
from pyvisim.retrieval.image_store import ExternalSearchIndex, InMemoryImageEmbeddingStore

faiss_index = faiss.IndexFlatIP(dim)
faiss.normalize_L2(vectors)
faiss_index.add(vectors)

store = InMemoryImageEmbeddingStore(
    gallery_paths,              # one path per indexed vector, same order
    embedder,                   # still needed, to embed the queries
    ExternalSearchIndex.from_faiss_index(faiss_index, name="flat-ip"),
)

Important

Normalization must be done by the user. An index built for METRIC_INNER_PRODUCT ranks by cosine similarity only if vectors are normalised before being added, and the embeddings of query images must be normalised the same way.
The scores stay the index’s own. An L2 index returns distances (lower is better), an inner-product index returns similarities (higher is better).

Since some external indexes are lossy, reconstruction is not always possible. In such cases, the original vectors must be passed explicitly to save_to_disk. The index itself cannot be written to disk either, so a rebuilt one is passed back to load_from_disk:

store.save_to_disk("gallery.safetensors", embeddings=original_embeddings)
restored = InMemoryImageEmbeddingStore.load_from_disk(
    "gallery.safetensors",
    search_index=ExternalSearchIndex.from_faiss_index(faiss_index, name="flat-ip"),
)

API reference¶

class pyvisim.retrieval.image_store.ExternalSearchIndex(index, vectors, *, name=None)[source]¶

Bases: object

A search index built elsewhere, adapted to the store’s interface.

Important

Whether a score is a distance (lower is better) or a similarity (higher is better) depends on the metric the wrapped index was built for, and any necessary normalization of the vectors (for a cosine ranking) must be done by the caller, before the index is built.

For more information, see the documentation: https://mechacritter.github.io/Python-Visual-Similarity/image_similarity_retrieval/image_store/external_search_index/external_search_index.html.

Parameters:

index (Any) – The index to search through. It must expose a search(queries, k) returning a (scores, ids) pair of (M, k) arrays whose ids are row numbers into vectors.
vectors (FloatNumpyArray) – The gallery vectors the index was built over, shape (N, D), in the order its ids refer to.
name (str | None) – Name identifying the index, kept across a save/load round trip so a store can be rebuilt on a matching one. If None, DEFAULT_EXTERNAL_NAME is used.

Raises:

AttributeError – If index has no search method.
ValueError – If vectors is not a non-empty 2-D matrix, or the index reports a size that does not match it.

classmethod from_faiss_index(index, vectors=None, *, name=None)[source]¶

Adapt a FAISS index, reading its vectors back when it can produce them.

An index that cannot reconstruct needs its vectors passed explicitly; so does one whose reconstruction this cannot catch, since a few index types abort the process instead of raising an error.

Normalization, if any, must be done by the caller. An index built for METRIC_INNER_PRODUCT only ranks by cosine similarity if the vectors were L2-normalised before they were added, and the queries handed to search() must be normalised the same way.

Parameters:

index (Any) – The FAISS index to search through.
vectors (ndarray[tuple[int, ...], dtype[floating[Any]]] | None) – The gallery vectors the index was built over, shape (N, D). Reconstructed from the index when omitted.
name (str | None) – Name identifying the index. If None, DEFAULT_EXTERNAL_NAME is used.

Returns:

An ExternalSearchIndex around index.

Raises:

ValueError – If vectors is omitted and the index cannot reconstruct them, or the index reports a different size.

Return type:

ExternalSearchIndex

search(query_vectors, k)[source]¶

Return the k nearest gallery vectors for each query vector.

The scores are whatever the wrapped index returns, unchanged.

Parameters:

query_vectors (ndarray[tuple[int, ...], dtype[floating[Any]]]) – A (D,) vector or an (M, D) batch of query vectors.
k (int) – Number of nearest neighbors to return per query.

Returns:

A (scores, ids) tuple of (M, k) arrays. ids are gallery row numbers, and missing neighbors are reported as -1.

Raises:

ValueError – If k is not a positive integer or the queries do not match the indexed dimensionality.

Return type:

tuple[ndarray[tuple[int, …], dtype[float32]], ndarray[tuple[int, …], dtype[int64]]]

vectors_at(ids)[source]¶

Read the gallery vectors stored under the given row numbers.

Parameters:: ids (Sequence[int] | ndarray[tuple[int, ...], dtype[int64]]) – Gallery row numbers, shape (n,), at least one.
Returns:: The (n, D) block of the requested vectors, read-only and in the given order.
Raises:: ValueError – If ids is empty, not one-dimensional, holds non-integers, or names a row outside the gallery.
Return type:: ndarray[tuple[int, …], dtype[float32]]

property dim: int¶: Dimensionality of the indexed vectors.

property index: Any¶: The wrapped index, as it was passed in.

property name: str¶: Name identifying the index across a save/load round trip.

property vectors: ndarray[tuple[int, ...], dtype[float32]]¶: The (N, D) gallery matrix the index was built over, read-only.