Pipeline

Pipeline glues several classical embedders into one. It embeds an image with every member, concatenates the per-member vectors, and compares the combined vectors with a single similarity function. Zhang et al. (2017), for example, found that combining VLAD and Fisher Vector embeddings improved fine-grained image recognition performance by 1.1% - 4.8% on Caltech-UCSD 2011 bird, FGVC-Aircraft, FGVC-Cars and Stanford dogs datasets.

from pyvisim.classic import FisherVectorEmbedder, Pipeline, VLADEmbedder

vlad = VLADEmbedder(n_clusters=64)
fisher = FisherVectorEmbedder(n_components=64)
for embedder in (vlad, fisher):
    embedder.learn(images)

pipeline = Pipeline([vlad, fisher], similarity_func="cosine")

vectors = pipeline.embed(images)     # (num_images, vlad_dim + fisher_dim)

score = pipeline.similarity_score(image1, image2)

pipeline.save_to_disk("pipeline.safetensors")   # Save the pipeline to disk

# Load the pipeline from disk
pipeline = Pipeline.load_from_disk("pipeline.safetensors")

References

API reference

class pyvisim.classic.Pipeline(embedders, similarity_func='cosine', *, normalize=True, batch_size=16)[source]

Bases: SerializableImageEmbedder

A pipeline for computing feature vectors using a set of embedders.

Parameters:

  • embedders (list[SerializableImageEmbedder]) – A list of SerializableImageEmbedder instances.

  • similarity_func (str) – Name of the built-in similarity metric to use. One of "cosine", "euclidean", "l1" or "manhattan".

  • normalize (bool) – Whether embed() L2-normalizes the joined embeddings it returns.

  • batch_size (int) – Maximum number of images processed in a single batch. Set to -1 to process all images as a single batch.

Raises:

ValueError – If embedders is empty or holds anything but a SerializableImageEmbedder.

embed(images, *, dims='HWC', value_range=(0.0, 255.0))

Embeds one or more images into a batch of vector representations.

Each image is normalized to a canonical uint8 (H, W, C) array before feature extraction, so NumPy arrays, torch tensors and other array-like inputs are all accepted. When a batch axis is present (via dims), every image in the batch is embedded. The resulting vectors are L2-normalized row by row when normalize is True.

Parameters:

  • images (_Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | Iterable[_Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes]]) – A single MatLike image, a batched array, or an iterable of images. Consider using an iterator for large datasets.

  • dims (str) – Axis-label string, one character per array axis in order: "H" = height (rows), "W" = width (columns), "C" = channels (e.g. RGB), "B" = batch size. For example, "HWC" is height × width × channels (NumPy/OpenCV single-image layout); "CHW" is channels × height × width (PyTorch single-image layout); "BCHW" is batch × channels × height × width (PyTorch batched layout). See pyvisim.typing.

  • value_range (tuple[float, float]) – The (low, high) range the input values live in; converted into the canonical [0, 255] range.

Returns:

vector representations of the given images, L2-normalized row by row if normalize is True.

Raises:

ValueError – If images holds no image.

Return type:

ndarray[tuple[int, …], dtype[floating[Any]]]

classmethod from_dict(state, **kwargs)[source]

Rebuilds the embedder a state dictionary describes.

Called on SerializableImageEmbedder itself, it hands the state to the from_dict of the class named under "__class__", so a state can be rebuilt without knowing which embedder wrote it.

Parameters:

  • state (dict[str, Any]) – A JSON-safe embedder description.

  • kwargs (Any) – Objects the state cannot describe, forwarded to the embedder’s own from_dict.

Returns:

The reconstructed embedder.

Raises:

  • ValueError – If state names no concrete embedder class.

  • NotImplementedError – If called on a subclass that does not implement its own from_dict.

Return type:

Pipeline

classmethod load_from_disk(path, **kwargs)

Loads an object previously saved with save_to_disk().

Not every part of an object survives serialization: an arbitrary callable such as a torchvision transform has no portable description, so it is left out of the file. Pass such an object back here as a keyword argument.

Parameters:

  • path (str | Path) – Path to the file to load.

  • kwargs (Any) – Objects the file cannot hold, forwarded to from_dict().

Returns:

A ready-to-use instance.

Raises:

  • FileNotFoundError – If path does not exist.

  • ValueError – If the file is not a valid file of this kind or was saved by a different class.

  • TypeError – If the class does not take one of kwargs.

Return type:

_SerializableT

save_to_disk(path)

Saves the serialized state of this object to a file.

Parameters:

path (str | Path) – Target file path. Overwritten if it exists.

Returns:

The path of the written file.

Raises:

OSError – If the destination directory does not exist.

Return type:

Path

set_batch_size(batch_size)

Sets the number of items processed per batch.

Parameters:

batch_size (int) – Maximum number of images processed in a single batch. Set to -1 to process all images as a single batch.

Raises:

ValueError – If batch_size is neither -1 nor a positive integer.

Return type:

None

similarity_score(images1, images2, *, dims='HWC', value_range=(0.0, 255.0))

Compute the similarity scores matrix between two (batches of) images.

Parameters:
Returns:

The similarity score matrix of shape (len(images1), len(images2)).

Return type:

ndarray[tuple[int, …], dtype[float32]]

to_dict()

Serializes this object into a JSON-safe state dictionary.

The mapping holds the output of _state() plus the format version under "format_version" and the class name under "__class__". Arrays may be embedded as __ndarray__ nodes, which the serialization layer stores as binary tensors.

Returns:

A JSON-safe description suitable for from_dict().

Return type:

dict[str, Any]

property normalize: bool

Whether the embeddings returned by embed() are L2-normalized.

property similarity_func: Callable[[ndarray[tuple[int, ...], dtype[floating[Any]]], ndarray[tuple[int, ...], dtype[floating[Any]]]], ndarray[tuple[int, ...], dtype[floating[Any]]]]

The resolved similarity function callable.

property similarity_func_name: str

The name of the configured similarity metric (e.g. "cosine").