Evaluation functions

Top-level entry points for scoring the quality of the images a store retrieves.

API reference

This module contains functions to evaluate the performance of a retrieval system.

pyvisim.eval.top_k_accuracy(images, image_labels, store, path_labels_dict, k)[source]

Computes top-k accuracy. For each query, we look at the top-k most similar results in the dataset. If any of them match the query’s label, that query is considered correct.

Parameters:
Returns:

Top-k accuracy (float) in the range [0, 1].

Return type:

float

pyvisim.eval.top_k_map(images, image_labels, store, path_labels_dict, k=None)[source]

Computes mean Average Precision over the queries, based on whether retrieved images have matching labels.

Parameters:

  • images (Iterable[_Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes]]) – Query images.

  • image_labels (Iterable[int]) – Corresponding labels for the query images.

  • store (EmbeddingStore) – An InMemoryImageEmbeddingStore (or any EmbeddingStore) holding the gallery embeddings and the embedder.

  • path_labels_dict (dict[str, int]) – dict {img_path: label}, covering every path of the store.

  • k (int | None) – Number of top results to consider. Each average precision is divided by the number of gallery images sharing the query label, capped at k.

Returns:

mAP

Return type:

float