BCESiameseNetwork

Both images are passed through the same shared-weight backbone and projection head. Each branch output is squashed with a sigmoid into a feature vector h in (0, 1)^D. The two branches are then combined by their component-wise L1 distance, and a single learned linear layer maps that distance vector to the probability of the pair showing the same class:

\[p(x_1, x_2) = \sigma\Bigl(\sum_{j} \alpha_j \, \bigl| h_{1,j} - h_{2,j} \bigr| + b\Bigr)\]

where the weights \(\alpha_j\) learn the importance of each feature dimension, so unlike ContrastiveSiameseNetwork the comparison metric itself is trained. The network is a binary classifier over pairs and is trained with binary cross-entropy on labels 1 (same class) / 0 (different class).

Following diagram visualizes this:

                  ┌──────────┐    ┌────────────────┐    ┌─────────┐
Input Image A ───►│ Backbone │───►│ Embedding Head │───►│ Sigmoid │───► Features A ──┐
                  └──────────┘    └────────────────┘    └─────────┘                  │
                       ╎                  ╎                  ╎                       │    ┌─────────┐    ┌───────────────┐
                       ╎                  ╎                  ╎                       ├───►│ |A - B| │───►│ Scoring Layer │───► P(same class)
                       ╎                  ╎                  ╎                       │    └─────────┘    └───────────────┘
                  ┌──────────┐    ┌────────────────┐    ┌─────────┐                  │
Input Image B ───►│ Backbone │───►│ Embedding Head │───►│ Sigmoid │───► Features B ──┘
                  └──────────┘    └────────────────┘    └─────────┘
                       (Shared Weights)

Example: training a BCE Siamese Network

See this tutorial.

API reference

class pyvisim.neural_networks.BCESiameseNetwork(backbone='resnet18', embedding_dim=128, transform=None, device='cpu', pretrained_backbone=True, *, batch_size=16)[source]

Bases: BackboneWithHead

Siamese network that classifies image pairs, proposed in Koch, G., Zemel, R., & Salakhutdinov, R. (2015). Siamese Neural Networks for One-shot Image Recognition.

For more information, see the documentation: https://mechacritter.github.io/Python-Visual-Similarity/neural_networks/bce_siamese/bce_siamese.html.

NOTE

The score is a learned probability, not a geometric similarity: it is symmetric in its inputs (the L1 distance is), lives in (0, 1), and for two identical images equals sigmoid(b) – the learned bias sets the operating point, so a perfect match does not score exactly 1.

References:

[1] Koch, G., Zemel, R., & Salakhutdinov, R. (2015). Siamese Neural Networks for One-shot Image Recognition. ICML Deep Learning Workshop. https://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf

param backbone:

name of feature-extraction network. See https://mechacritter.github.io/Python-Visual-Similarity/neural_networks/backbones/backbones.html.

param embedding_dim:

Dimensionality of the twin feature vectors that the scoring layer compares.

param transform:

processing transform applied to every input image. If None, the ImageNet preprocessing matching the backbone is used.

param device:

Device on which the model is placed.

param pretrained_backbone:

Whether to use a backbone pretrained on ImageNet. If you are loading the BCESiameseNetwork from a checkpoint, set this to False to avoid downloading the weights again.

param batch_size:

Maximum number of images processed in a single batch. Set to -1 to process all images as a single batch.

raises ValueError:

If embedding_dim is not a positive integer or if backbone is not a supported backbone name.

embed(images, *, dims='HWC', value_range=(0.0, 255.0))[source]

Not implemented for this class. Please do not use!

Parameters:
Return type:

ndarray[tuple[int, …], dtype[floating[Any]]]

forward(x1, x2)[source]

Computes same-class logits for a batch of aligned image pairs.

The i-th logit scores the pair (x1[i], x2[i]); apply torch.sigmoid() to obtain probabilities, or feed the logits directly to torch.nn.BCEWithLogitsLoss during training.

Parameters:

  • x1 (Tensor) – First preprocessed image batch, shape (batch, channels, H, W).

  • x2 (Tensor) – Second preprocessed image batch of the same shape.

Returns:

Logit tensor of shape (batch,).

Raises:

ValueError – If the two batches differ in shape.

Return type:

Tensor

classmethod from_dict(state, **kwargs)

Rebuilds the embedder a state dictionary describes.

Called on SerializableImageEmbedder itself, it hands the state to the from_dict of the class named under "__class__", so a state can be rebuilt without knowing which embedder wrote it.

Parameters:

  • state (dict[str, Any]) – A JSON-safe embedder description.

  • kwargs (Any) – Objects the state cannot describe, forwarded to the embedder’s own from_dict.

Returns:

The reconstructed embedder.

Raises:

  • ValueError – If state names no concrete embedder class.

  • NotImplementedError – If called on a subclass that does not implement its own from_dict.

Return type:

_NeuralEmbedderT

classmethod load_from_disk(path, **kwargs)

Loads an object previously saved with save_to_disk().

Not every part of an object survives serialization: an arbitrary callable such as a torchvision transform has no portable description, so it is left out of the file. Pass such an object back here as a keyword argument.

Parameters:

  • path (str | Path) – Path to the file to load.

  • kwargs (Any) – Objects the file cannot hold, forwarded to from_dict().

Returns:

A ready-to-use instance.

Raises:

  • FileNotFoundError – If path does not exist.

  • ValueError – If the file is not a valid file of this kind or was saved by a different class.

  • TypeError – If the class does not take one of kwargs.

Return type:

_SerializableT

save_to_disk(path)

Saves the serialized state of this object to a file.

Parameters:

path (str | Path) – Target file path. Overwritten if it exists.

Returns:

The path of the written file.

Raises:

OSError – If the destination directory does not exist.

Return type:

Path

set_batch_size(batch_size)

Sets the number of items processed per batch.

Parameters:

batch_size (int) – Maximum number of images processed in a single batch. Set to -1 to process all images as a single batch.

Raises:

ValueError – If batch_size is neither -1 nor a positive integer.

Return type:

None

similarity_score(images1, images2, *, dims='HWC', value_range=(0.0, 255.0))[source]

Compute the similarity scores matrix between two (batches of) images.

Parameters:
Returns:

The similarity score matrix of shape (len(images1), len(images2)).

Return type:

ndarray[tuple[int, …], dtype[floating[Any]]]

to_dict()

Serializes this object into a JSON-safe state dictionary.

The mapping holds the output of _state() plus the format version under "format_version" and the class name under "__class__". Arrays may be embedded as __ndarray__ nodes, which the serialization layer stores as binary tensors.

Returns:

A JSON-safe description suitable for from_dict().

Return type:

dict[str, Any]

property backbone: Module

The shared feature-extraction backbone.

property device: device

The device the model’s parameters live on.

Derived from the parameters themselves rather than cached, so it stays correct after the user moves the model with model.to(...).

property head: Module

The projection head mapping backbone features to embeddings.

property normalize: bool

Whether the embeddings returned by embed() are L2-normalized.

property scorer: Module

The learned layer mapping L1 distances to same-class logits.

property similarity_func: Callable[[ndarray[tuple[int, ...], dtype[floating[Any]]], ndarray[tuple[int, ...], dtype[floating[Any]]]], ndarray[tuple[int, ...], dtype[floating[Any]]]]

The resolved similarity function callable.

property similarity_func_name: str

The name of the configured similarity metric (e.g. "cosine").

Parameters:

  • backbone (str)

  • embedding_dim (int)

  • transform (Compose | None)

  • device (str | device)

  • pretrained_backbone (bool)

  • batch_size (int)