MSSSIM

MS-SSIM goes one step further than SSIM by computing the SSIM at multiple scales. Hence, images are first downsampled by half (up to 5 times) and the SSIM is computed at each scale, then aggregated. Given two images x and y, MS-SSIM(x, y) is defined as:

\[\text{MS-SSIM}(x, y) = \left[l_M(x, y)\right]^{\alpha_M} \cdot \prod_{j=1}^{M} \left[c_j(x, y)\right]^{\beta_j} \left[s_j(x, y)\right]^{\gamma_j}\]

where:

  • \(l_j\), \(c_j\) and \(s_j\) are the luminance, contrast and structure components at scale \(j\), as defined in the SSIM formula.

  • \(M\) is the coarsest scale, which is set by the number of entries in weights.

  • \(\alpha_M\), \(\beta_j\) and \(\gamma_j\) are the exponents that weight the contribution of each scale. In pyvisim, the default weights, proposed by Wang et al. (2003), are used.

At the end of each scale, the images are low-pass filtered and downsampled by a factor of two through 2x2 average pooling. The luminance component is evaluated at the coarsest scale \(M\) only, so the product above is computed from the per-scale contrast-structure map means \(cs_j\) and the full SSIM mean \(\text{ssim}_M\):

\[\text{MS-SSIM}(x, y) = \text{ssim}_M^{\,w_M} \cdot \prod_{j=1}^{M-1} cs_j^{\,w_j}\]

Each entry of weights is one exponent \(w_j\), ordered with the coarsest scale last, and the number of entries sets the number of scales.

The Gaussian window has to fit the images after n_scales - 1 halvings, so each image side has to measure at least window_size * 2 ** (n_scales - 1) pixels.

Usage

from pyvisim.dense.structural import MSSSIM

msssim = MSSSIM(batch_size=16)
matrix = msssim.similarity_score(gallery, queries)   # (N, M) matrix

Benchmark

Important

This file was generated by the script docs/dense/structural/benchmarks/generate_benchmark.py. Do not edit manually!

All images are drawn from the Oxford Flower dataset (train split, seed 0), with a disjoint image subset per experiment and per metric. num_workers=4. 7 timed calls after one warm-up.

Baseline: torchmetrics 1.9.0 — torchmetrics.functional.image.multiscale_structural_similarity_index_measure(data_range=255.0, kernel_size=11, sigma=1.5, normalize=’relu’).

Accuracy

Each image (native resolution) is scored against a copy distorted with Gaussian noise (std 15). The error is the absolute difference between the pyvisim score and the baseline score.

Image

pyvisim

Baseline

Abs. error

image_00852.jpg

0.820745

0.820213

5.33e-04

image_01870.jpg

0.844088

0.841904

2.18e-03

image_01927.jpg

0.861056

0.860066

9.90e-04

image_02102.jpg

0.889506

0.888125

1.38e-03

image_02881.jpg

0.877060

0.877032

2.75e-05

image_03035.jpg

0.850458

0.850367

9.07e-05

image_05038.jpg

0.854924

0.854978

5.38e-05

image_06951.jpg

0.854107

0.854030

7.69e-05

Runtime

Median wall-clock time per full scoring call (MS-SSIM of every image in the first gallery against every image in the second). For a fair comparison, GPU is disabled.

Scenario

Images

pyvisim (ms)

Baseline (ms)

Speed-up

1 pair (512x512), 512x512

image_00037.jpg, image_06391.jpg

25.3

76.3

3.0x

1 expanded pair (1024x1024), 1024x1024

image_00191.jpg

121

325

2.7x

batch of 4 (16 pairs), 256x256

image_00018.jpg, image_04460.jpg, image_00547.jpg, image_08160.jpg

98.8

323

3.3x

batch of 8 (64 pairs), 256x256

image_03502.jpg, image_04084.jpg, image_05542.jpg, image_00609.jpg, image_04044.jpg, image_07394.jpg, image_05554.jpg, image_02798.jpg

369

1,360

3.7x

MS-SSIM median runtime

API reference

class pyvisim.dense.structural.MSSSIM(weights=(0.0448, 0.2856, 0.3001, 0.2363, 0.1333), window_size=11, sigma=1.5, k1=0.01, k2=0.03, batch_size=16, num_workers=None)[source]

Bases: DenseMetricBase

Multi-Scale Structural Similarity index (Wang et al., 2003).

For more information, see the documentation: https://mechacritter.github.io/Python-Visual-Similarity/dense/structural/ms_ssim/ms_ssim.html.

NOTE

Scale means below zero are clamped to zero before they are raised to a fractional power, which keeps the result real. Howevcer, natural image pairs are rarely affected by the clamping.

param weights:

One positive exponent per scale, coarsest scale last. The number of weights sets the number of scales. WANG_WEIGHTS holds the five exponents calibrated in the original paper.

param window_size:

Side length of the Gaussian window, an odd integer >= 3.

param sigma:

Standard deviation of the Gaussian window.

param k1:

Luminance stabilization constant.

param k2:

Contrast stabilization constant.

param batch_size:

Maximum number of image pairs processed in a single batch. Set to -1 to process all images as a single batch.

param num_workers:

Number of threads to use for the computation. If None, the PYVISIM_NUM_THREADS environment variable decides, which can also be changed through os.environ["PYVISIM_NUM_THREADS"].

raises ValueError:

If any parameter is outside its valid range.

set_batch_size(batch_size)

Sets the number of items processed per batch.

Parameters:

batch_size (int) – Maximum number of images processed in a single batch. Set to -1 to process all images as a single batch.

Raises:

ValueError – If batch_size is neither -1 nor a positive integer.

Return type:

None

similarity_score(image1, image2, *, dims='HWC', value_range=(0.0, 255.0))

Compute the pairwise score matrix between two image batches.

Every image is normalized to the canonical uint8 (H, W[, C]) layout in [0, 255] first, so the metric always operates on the same value scale regardless of the input dtype or range.

Parameters:
Returns:

A (N, M) matrix scoring every image of image1 against every image of image2.

Raises:

  • InvalidImageError – If an input cannot be converted to a numeric array.

  • ValueError – If a batch is empty, the two batches hold images of different shapes, or the images are too small for the metric.

Return type:

ndarray[tuple[int, …], dtype[floating[Any]]]

property k1: float

Luminance stabilization constant.

property k2: float

Contrast stabilization constant.

property n_scales: int

Number of pyramid scales, one per weight.

property sigma: float

Standard deviation of the Gaussian window.

property weights: tuple[float, ...]

Per-scale exponents, coarsest scale last.

property window_size: int

Side length of the Gaussian window, in pixels.

Parameters: