PaperlayerView exact revision on arXiv

Source-linked AI summary

JLD: Perceptual Distance Through A Jacobian Lens

Shreshth Saini, Balu Adsumilli, Alan C. Bovik

arXiv:2610.05967v1cs.CVcs.AI

TL;DR

Pixel error does not reflect human judgments well, and perceptual metrics fitted to those judgments can lose correlation when resolution changes. JLD uses a frozen encoder’s Jacobian to weight early image features, achieving strong agreement across image databases while remaining resolution-stable.

  • Problem

    Pixel error treats all changes equally despite differing perceptual severity, while human-fitted perceptual distances can lose correlation when image resolution changes.

    Source: §1 · p. 2

  • Method

    JLD weights spatially detailed early features by their sensitivity to a frozen vision encoder’s output, forming a fixed metric without human labels.

    Source: §1 · p. 2

  • Results

    JLD achieves the highest mean agreement with human judgments among the evaluated distances across image databases and remains stable as resolution doubles.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    JLD’s fixed geometry makes individual predictions directly inspectable through maps and contours.

    Source: §6 · p. 10

    JLD is weaker on restoration outputs (0.624 SRCC on PIPAL) and global contrast changes (0.378 on TID2013).

    Source: N LIMITATIONS · p. 33

Abstract

from arXiv · show

Image compression, restoration, and generation all require a way to measure how different two images look to a person. Pixel error ignores how people see, while the most accurate perceptual distances are typically fitted to human judgments, tying them to a fixed data and resolution. For example, when image resolution is doubled, the correlation of DISTS with human scores on TID2013 drops from 0.815 to 0.717. We introduce the Jacobian Lens Distance (JLD), which derives its perceptual geometry from a frozen vision encoder rather than from human labels. JLD combines the locality of early patch features with the perceptual sensitivity captured by later encoder representations. Specifically, we use the encoder Jacobian to identify directions in the early feature space that most strongly affect the encoder output, producing a fixed metric tensor, $E[J^\top J]$, which we call the Jacobian lens. The lens is fitted only once from 100 unlabeled images, taking about 35 seconds. Locally, this construction defines a pullback metric in pixel space, giving JLD a clear geometric interpretation that can be directly analyzed on real images. Across four standard perceptual databases, JLD achieves state-of-the-art performance and consistently outperforms LPIPS, DISTS, PieAPP, and DreamSim. JLD is also robust to changes in image resolution, on TID2013, its lens-term correlation remains nearly unchanged when the resolution is doubled, decreasing only from 0.850 to 0.845. We further introduce JLD-fast, which is $4\times$ faster than LPIPS-VGG while achieving a mean correlation of 0.911. Finally, JLD naturally extends to video, reaching a correlation of 0.786 on Waterloo IVC 4K compared with 0.611 for VMAF.

1 INTRODUCTION

JLD addresses the mismatch between pixel error and human perception by weighting early image features according to a frozen encoder’s output sensitivity. It reports strong benchmark performance and stability to resolution changes without fitting its lens to human judgments.

1 INTRODUCTION

  • 79.9% of equal-PSNR TID2013 pairs are ranked in line with human preference by JLD, versus 51.1% for pixel error.The cited examples show that similar PSNR values can accompany very different perceptual severity.Source: §1 · p. 2
  • Pixel error weights every change equally, unlike human vision, while human-fitted perceptual distances can lose correlation when resolution changes.For TID2013, DISTS drops from 0.815 to 0.717 SRCC when resolution rises from 256 to 512 pixels.Source: §1 · p. 2
  • JLD combines early features’ spatial detail with weights based on their effect on encoder output, reaching 0.850 SRCC versus 0.784 for raw features and 0.742 for output tokens.The weighted features also reproduce the human ordering of five example distortions, unlike raw features (τ = 0.2).Source: §1 · p. 2
  • The Jacobian lens is a fixed projection onto 64 leading directions of the encoder-sensitivity metric; those directions account for 76.5% of its trace, without human judgments.The metric is averaged over natural images and defines a pseudometric whose local form is a pullback metric in pixel space.Source: §1 · p. 2
  • Across 15 full-reference distances, JLD achieves state-of-the-art performance, beats every learned deep distance, and remains stable as image resolution doubles.JLD-fast is four times faster than LPIPS-VGG, and a single fitted lens transfers across 14 encoders and extends to video.Source: §1 · p. 3

2 BACKGROUND

Prior perceptual distances choose feature weights through learned human judgments, equal weighting, or models of early vision. JLD instead derives a shared feature-space tensor from a frozen encoder’s output sensitivity.

2 BACKGROUND

  • Feature distances differ mainly in their choice of Q: raw feature distances weight channels equally, LPIPS learns diagonal weights from human judgments, and DISTS fits feature-statistic weights.DreamSim fine-tunes its encoder on human triplets.Source: §2 · p. 3
  • Classical perceptual metrics derive image-dependent pixel-space tensors from early-vision transforms or response-model Fisher information.Eigen-distortions in the latter predict the most and least visible directions.Source: §2 · p. 3
  • JLD differs from prior pullback metrics by deriving its tensor from a frozen encoder’s own output and fixing it on early features for use across images.Other cited approaches use active subspaces, decoder-induced or language-model metrics, or averaged intermediate-to-final Jacobians through vocabulary.Source: §2 · p. 3

3 THE JACOBIAN LENS DISTANCE

JLD uses early patch features for spatial detail and an encoder-output Jacobian to determine which feature directions matter. Its fixed lens yields a distance with a local pullback geometry in pixel space.

3.1 ENCODER AND FEATURES

  • After one attention block, DINOv2-S patch tokens retain local structure while incorporating context; block 1 gives the largest TID2013 agreement gain, reaching 0.850.The same lens applies at every image size because images are center-cropped to a multiple of 14 pixels.Source: §3.1 · p. 3

3.2 SENSITIVITY AS A METRIC TENSOR

  • Averaging squared Jacobian responses over images and positions yields a fixed sensitivity matrix; its 64 leading directions retain 76.5% of the trace and are shared across images, positions, and resolutions.The fit uses 100 unlabeled DIV2K images and takes 35 seconds on one A100 GPU.Source: §3.2 · p. 4

3.3 THE DISTANCE

  • JLD combines a chroma-blurring pre-filter with complementary lens and whole-output terms; the output term captures global changes that might be too diffuse for the lens term.Development data set the output-term weight to 1/2; JLD-fast keeps the pre-filter and lens term but stops the encoder after block 1.Source: §3.3 · pp. 4–5
  • Locally, the lens term is a quadratic form in pixel space, with an image-dependent pullback G; Figure 2 contrasts constant-JLD and constant-MSE contours on equal-pixel-energy planes.The predicted pullback is shown as a dashed ellipse.Source: §3.3 · p. 5, Figure 2 · p. 5

4 GEOMETRY OF THE DISTANCE

JLD induces an image-space geometry that can be analyzed directly: its local sensitivity is anisotropic and concentrated on encoder-selected feature directions. The lens term discounts feature motion the encoder output ignores, unlike raw feature distance.

4 GEOMETRY OF THE DISTANCE

  • Across 171.4 million same-reference triplets, the lens term satisfies the triangle inequality; the full distance violates it in at most 107 triplets per million, with a maximum violation of 3.6%.A lens fitted to one image has median 0.72 overlap with the shipped lens, comparable to 0.69 between two fits of the same image using different methods.Source: §4 · p. 5

4.1 THE LOCAL SHAPE OF THE DISTANCE

  • Locally, JLD’s pullback metric maps pixel perturbations to ellipsoidal distance contours, with large-eigenvalue directions registered strongly and small-eigenvalue directions tolerated more.MSE instead has spherical contours because its local metric is proportional to the identity.Source: §4.1 · p. 5
  • At least 89% of pixel directions leave the lens term unchanged to first order, because each patch contributes rank at most 64 while containing 588 pixel values.A pixel change can be nearly invisible to JLD if it barely moves tokens or mostly moves them along directions the output ignores; the latter case is more common.Source: §4.1 · pp. 5–6
  • Around I19, JLD’s equal-energy distortion contours are 2.33 times longer along chroma than blur, versus 1.06 for MSE, 1.61 for LPIPS-VGG, and 1.64 for DISTS.Across three equal-MSE rings, JLD varies by factors of 2.4, 1.5, and 4.4; its local metric is also smaller in textured regions.Source: §4.1 · p. 6

4.2 DISTORTIONS THROUGH THE LENS

  • At level 5 on I19, JLD is 0.516 for JPEG and 0.129 for colour noise, matching their human scores of 1.66 and 4.47 despite similar PSNR.The lens retains 14.3% of JPEG displacement but 6.4% of noise displacement; across 120 distortions of I19 it retains 9.6% of displacement energy.Source: §4.2 · p. 6

5 PREDICTING HUMAN JUDGEMENTS

JLD is evaluated against human judgments across image and video benchmarks, with tests of its geometry, component choices, resolution stability, and computational cost. It leads the four-database mean among evaluated image distances and performs strongly on pairwise judgments, while extending to video.

5.1 IMAGE SIMILARITY BENCHMARKS

  • 0.926 mean SRCC over TID2013, CSIQ, LIVE, and KADID-10k gives JLD the highest score among evaluated distances, ahead of VSI at 0.915, DeepWSD at 0.913, DeepDC at 0.905, and DISTS at 0.900.JLD exceeds every learned deep distance on all four datasets without human-label fitting; every reported gap has a 95% interval excluding zero.Source: Table 1 · p. 7, §5.1 · p. 7
  • 90.0% of 699,534 same-reference choices are ordered correctly by JLD, the highest among 17 distances, ahead of DeepDC at 89.4% and DISTS at 88.4%.Among 24,623 pairs with PSNR differences under 0.25 dB, JLD agrees with people on 76.0%, versus 74.4% for DISTS and 65.3% for LPIPS-VGG.Source: Figure 4 · p. 7, §5.1 · p. 8

5.2 SEEING THE DISTANCE

  • On held-out KADID-10k, distortions selected at equal JLD have human-score standard deviation 0.46 MOS, while equal-PSNR selections have 1.26.For another reference, JLD ordering reaches 0.921 SRCC across 121 distortions, versus 0.566 for PSNR.Source: Figure 5 · p. 8, §5.2 · p. 8

5.3 ABLATION

  • 0.926 four-set mean SRCC for full JLD exceeds 0.772 for raw block-1 features, 0.851 for 64 PCA directions, and 0.905 for 64 lens directions before the chroma pre-filter and output term.The gain comes from which directions are selected rather than their number; across 14 frozen encoders, a fitted lens improves over raw features from the same block.Source: Figure 6 · p. 9, §5.3 · p. 9

5.4 RESOLUTION, COST AND VIDEO

  • From 256 to 512 pixels, the TID2013 lens-term SRCC changes from 0.850 to 0.845, while unweighted block-1 features fall from 0.784 to 0.656, DISTS from 0.815 to 0.717, and LPIPS-VGG from 0.757 to 0.661.On CSIQ, the lens rises from 0.923 to 0.958, the highest score among 15 methods at 512 pixels.Source: Figure 7 · p. 9, §5.4 · p. 9
  • JLD-fast achieves 0.911 mean SRCC at 2.5 ms per pair on an A100, 4.3 times faster than LPIPS-VGG and 4.0 times faster than DISTS.On Waterloo IVC 4K, JLD-video reaches 0.786 SRCC versus 0.611 for VMAF; on held-out AVT-VQDB-UHD-1 it reaches 0.912, within 0.020 of VMAF.Source: Figure 7 · p. 9, §5.4 · p. 9

6 DISCUSSION AND CONCLUSION

JLD derives perceptual weights from a frozen encoder’s output sensitivity, combining early features with later representations. The resulting fixed geometry achieves strong image-database agreement, remains stable across resolution changes, and supports direct inspection of predictions.

6 DISCUSSION AND CONCLUSION

  • A frozen encoder’s output Jacobian supplies a fixed metric tensor for weighting early features, and the lens improves over raw features for all 14 tested encoders.The lens is fitted in about 35 seconds from 100 unlabelled images; JLD-fast evaluates a pair in 2.5 ms.Source: §6 · p. 10
  • Across image databases, JLD has the highest mean agreement among the evaluated distances and remains stable when image resolution doubles.Its fixed geometry also makes individual predictions directly inspectable through maps and contours.Source: §6 · p. 10

A METHOD DETAILS

JLD uses early patch tokens from a frozen DINOv2-S encoder and weights their changes with a lens calibrated from output sensitivity. The resulting fixed projection supports image-size flexibility, interpretable geometry, and faster or domain-specific variants.

A.1 ENCODER AND PATCH FEATURES

  • JLD reads 384-dimensional patch tokens after block 1 of frozen DINOv2-S and uses the final class token as the output whose sensitivity defines the metric.The same fitted lens applies at different image sizes by center-cropping to patch-grid multiples and interpolating positional embeddings.Source: §A.1 · p. 14

A.2 CALIBRATING THE LENS

  • The lens calibration averages squared output responses to patch-token changes, estimates the sensitivity matrix with Gaussian output probes, and retains its 64 leading eigenvectors.The fit uses 100 unlabelled DIV2K images, no image pairs, distortions, or human labels, and takes 34.53 seconds on one A100.Source: §A.2 · pp. 14–15

A.3 FEATURE RANK

  • The leading 64 of 384 sensitivity directions contain 76.5% of the trace, while the lens retains only 33.7% of feature variance in the reported diagnostic.Development SRCC peaks at 0.913 for 64 lens directions, versus 0.782 for all 384 directions and at most 0.883 for PCA subspaces.Source: §A.3 · p. 16

A.4 LENS SENSITIVITY

  • Across 153 perturbations, RMS lens displacement correlates 0.643 with final class-token change, versus 0.147 for raw block-1 displacement.At block 0, lens directions act as matched filters on RGB patch changes, forming the basis of JLD-lin.Source: §A.4 · p. 16

A.5 CHROMA PRE-FILTER

  • A fixed pre-filter blurs chroma before both images are evaluated on the same patch grid, addressing the unfiltered lens’s residual high-frequency colour responses.At pixel Nyquist frequency, red–green and blue–yellow responses retain 0.208 and 0.274 of their peaks, respectively.Source: §A.5 · pp. 16–17

A.6 POOLING AND OUTPUT TERM

  • JLD quadratically pools per-patch lens lengths and adds half the cosine distance between final class tokens as a whole-image output term.The output term raises CSIQ correlation from 0.956 to 0.971; weights from 0.25 to 1 keep CSIQ between 0.968 and 0.971 and TID2013 between 0.872 and 0.877.Source: §A.6 · p. 17

A.7 VARIANTS: JLD-FAST, JLD-LIN AND JLD-VIDEO

  • JLD-fast removes the output term and stops after block 1, reducing cost from 118.5 GFLOPs and 12.2 ms to 10.7 GFLOPs and 2.5 ms per pair.JLD-lin uses 64 fixed RGB patch filters, while JLD-video uses a 16-dimensional lens on space-time tokens from frozen VideoMAE-Base.Source: §A.7 · p. 17

A.8 FIT STABILITY; A.9 IMPLEMENTATION

  • Ten calibration images yield 0.877 overlap with the 100-image lens and change development correlation by less than 0.002; changing the output target instead gives 0.510 overlap.Inference uses no gradients, and the implementation center-crops images, applies the chroma pre-filter, and scores with the encoder and fitted lens.Source: §A.8 · p. 17, §A.9 · p. 18

B.1 THE SENSITIVITY MATRIX AND THE LENS

The section characterizes the Jacobian lens as a sensitivity-weighted feature-space projection and analyzes the distance’s geometry, estimation, and behavior across images. It also reports conditions and practical caveats for interpreting that geometry.

B.1 THE SENSITIVITY MATRIX AND THE LENS

  • The top-k lens maximizes captured encoder sensitivity, while the local pixel-space form is a pullback of the fixed feature-space metric.For the released lens, an equal-norm displacement outside the lens has at most 0.173 of the leading eigendirection’s RMS sensitivity; real distortions show a pooled ratio of 5.7.Source: §B.1 · pp. 18–19

3. Over random inputs,

  • Random-probe estimation is unbiased, and the stated stability bound needs no eigengap; two independent released-lens refits overlap by 0.962.The released calibration uses 400 crops and eight probes per crop; measured refit variation is 8.4% of the tensor norm, with at most 0.35% sensitivity loss between fitted lenses.Source: §B.1 · p. 19, §3 · p. 19

B.2 THE LENS TERM AS A DISTANCE

  • With a fixed patch grid and lens, the lens term is a pseudometric, but distinct images can have zero distance when their projected patch features agree.A crop chosen independently for each image pair does not preserve the single-map condition needed for this guarantee.Source: §B.2 · p. 20

B.3 REFINING THE PIXEL GRID

  • For JLD-lin, the refined-grid limit depends on filter DC gains, which contain 0.28% of total filter energy; block-1 resolution stability is empirical, not covered by the theorem.The theorem assumes a Lipschitz image difference and a non-saturating pre-filter.Source: §B.3 · p. 21

B.4 ONE LENS FOR EVERY IMAGE

  • The global rank-k lens maximizes sensitivity averaged over images, with a median image-wise sensitivity loss of 5.0% relative to image-specific optima.The loss is controlled by the difference between the image-specific and global sensitivity matrices.Source: §B.4 · p. 21, §2 · p. 22

B.5 THE OUTPUT TERM AND THE TRIANGLE INEQUALITY

  • The lens term is a pseudometric, but the full JLD distance is not guaranteed to satisfy the triangle inequality.A Euclidean norm over concatenated lens features and a suitably scaled unit output restores the guarantee and gives similar rankings.Source: §B.5 · p. 22

D.1 ISO-DISTANCE CONTOURS: BRIGHTNESS AGAINST NOISE

  • At equal RMS pixel energy, JLD stretches brightness-versus-noise contours by 2.78, while MSE anisotropy is 1.02 on the reported TID2013 plane.At fixed JLD distance 0.08, PSNR ranges from 19.6 dB for brightness change to 32.6 dB for noise.Source: §D.1 · p. 23

D.2 ONE LENS SERVES EVERY IMAGE

  • The released lens overlaps one-image lenses by a median 0.72, while independent refits capture more sensitivity than the first one-image fit for 103 of 125 images.Pooling the DIV2K one-image estimates recovers the released lens with 0.954 subspace overlap; the authors attribute much of the discrepancy to estimation noise.Source: §D.2 · p. 23, Figure A6 · p. 23

D.3 THE METRIC TENSOR FIELD

The metric tensor field is examined on real images, then JLD is compared with human preferences across KADID-10k distortion examples. JLD tracks human scores well across distortion families, though systematic errors remain.

D.3 THE METRIC TENSOR FIELD

  • Across 234 TID2013 image cells, the metric-tensor glyph radius falls with local contrast (Spearman −0.86), and smooth regions are 2.3 times as sensitive as the most textured quarter.In 91% of cells with a clear gradient, the stiff axis lies more than 60° from the gradient direction.Source: §D.3 · p. 24

E.1 2-AFC

  • JLD agrees with human preference on 93.3% of trials, versus 93.0% for DISTS, 82.9% for LPIPS-VGG, 81.2% for PSNR, and 78.4% for SSIM.JLD alone agrees with humans on 1,954 pairs, compared with 1,418 pairs showing the reverse pattern.Source: §E.1 · p. 25

E.2 CONSISTENCY ACROSS DISTORTION FAMILIES

  • Across 125 distortion-family–level cells, JLD reaches SRCC 0.940 with 0.38 MOS linear-fit error, while DISTS reaches 0.931 with 0.40.JLD under-penalises colour block distortion by −0.72 MOS and colour shift by −0.53, and over-penalises high sharpening by +0.51.Source: §E.2 · p. 26

E.3 GMAD STRESS TESTS

  • In gMAD tests against LPIPS-VGG, JLD wins at all nine operating points with a mean human-score gap of 2.20 MOS.Against DISTS, each metric wins as attacker at six of nine operating points.Source: §E.3 · p. 26, Figure A10 · p. 26

F FULL IMAGE RESULTS

Full-image results show strong JLD performance across several image sets and distortion types, but it is not the strongest method on every dataset or subset.

F FULL IMAGE RESULTS

  • JLD reaches SRCC 0.971 on CSIQ development, 0.964 on LIVE, and 0.892 on KADID-10k test, but DeepDC remains stronger on KADID-10k and PIPAL, while VSI remains stronger on TID2013.JLD reaches SRCC 0.877 on TID2013, 0.603 on PIPAL public, and 0.624 on PIPAL validation.Source: F FULL IMAGE RESULTS · p. 26, Table A3 · p. 27, Table A4 · p. 27
  • On BAPPS, JLD reaches agreement scores of 0.821 on CNN-generated distortions and 0.681 on traditional distortions, while LPIPS-VGG reaches 0.814 and 0.714, respectively.JLD also reaches 0.625 on colour changes, 0.595 on deblurring, 0.608 on frame interpolation, and 0.694 on super-resolution.Source: F FULL IMAGE RESULTS · p. 27
  • The no-reference extension reaches 0.486 on held-out KonIQ images, while CLIP-IQA reaches 0.685 on the full dataset; these results use different subsets and should not be directly compared.Source: F FULL IMAGE RESULTS · p. 28

G.1 ABLATIONS

Ablations, resolution tests, and perceptual probes examine which components of JLD carry its agreement with human judgments. The results also test its transfer across encoders and video.

G.1 ABLATIONS

  • On four datasets, the 64 leading eigenvectors of the Jacobian metric reach mean SRCC 0.905, versus 0.851 for leading PCA directions and 0.772 for all 384 raw features.Random 64-direction projection leaves the raw-feature mean unchanged; the output term alone reaches 0.826, indicating that most signal comes from the lens term. Figure A11 also shows a same-PSNR pair where JLD ranks images as people do while DISTS and LPIPS-VGG reverse the preference.Source: §G.1 · p. 28, Figure A11 · p. 28

G.2 RESOLUTION RESULTS

  • On TID2013, the lens-term correlation changes only from 0.850 at 256 pixels to 0.845 at 512 pixels, while DISTS falls from 0.815 to 0.717.Unweighted block-1 features fall from 0.784 to 0.656, and LPIPS-VGG from 0.757 to 0.661.Source: §G.2 · p. 29, Table A6 · p. 29

H DEVELOPMENT CHOICES

  • The selected CLS-target configuration reaches development SRCC 0.915, compared with 0.916 for the best tested configuration.Block 1 and rank 64 are selected; pooled natural-image sensitivity reaches 0.861 SRCC on 1,000 TID2013 pairs, versus 0.679 for per-reference sensitivity.Source: H DEVELOPMENT CHOICES · p. 29, Figure A13 · p. 30, Table A7 · p. 30

I WHAT THE LENS SEES IN GRATINGS

  • Grating probes show that the lens reproduces lower sensitivity to fine oblique gratings and texture masking, but does not reproduce the human luminance contrast-sensitivity function.Its luminance response peaks at a lower spatial frequency, and the unfiltered lens remains too sensitive to the finest chromatic patterns.Source: I WHAT THE LENS SEES IN GRATINGS · p. 30, Figure A14 · p. 31

J.1 HUMAN STUDY

  • Across trials with an untied majority, JLD agrees with the human majority on 24 of 28 decisions, compared with 14 of 26 for DISTS.On 16 rated-image-pair trials, the majority agrees with MOS on all 16; JLD agrees on 12 and DISTS on four.Source: §J.1 · p. 31, Figure A15 · p. 31

J.2 METAMERS AND TARGETED PERTURBATIONS

  • At nominal 40 dB PSNR, lens-maximising perturbations increase lens distance by a factor of 134, while LPIPS-VGG changes by only 1.23.Approximate metamers remain close under the unfiltered lens, with median ratio 1.23 relative to random perturbations, while LPIPS-VGG rises to 14.5.Source: §J.2 · p. 31, Figure A16 · p. 32

K ENCODERS

  • The lens improves over raw features for all 14 frozen encoders and over PCA 64 for 13 of 14, while DINOv2-S gives the strongest development result.VideoMAE K400 reaches SRCC 0.786 with a 16-dimensional lens, versus 0.771 for PCA 64 and 0.710 for full features.Source: Figure A12 · p. 29, K ENCODERS · p. 32, Table A8 · p. 32

L VIDEO

  • On Waterloo IVC 4K, JLD-video reaches SRCC 0.786 versus 0.611 for VMAF; on held-out AVT, it reaches 0.912 versus 0.932 for VMAF.JLD-video is the only one of the four compared methods above 0.78 on both datasets.Source: L VIDEO · p. 32, §L.1 · p. 33, Table A9 · p. 33

M MATCHED PIXEL ERROR

Matched-pixel-error examples show both successes and failures of JLD against human preferences.

M MATCHED PIXEL ERROR

  • For same-reference pairs with similar pixel error, JLD agrees with human preferences in the top two examples but reverses them in the bottom two.Source: M MATCHED PIXEL ERROR · p. 33

N LIMITATIONS

JLD has weaker results on restoration outputs and global contrast changes, and its lens and combined distance have geometric caveats. Its evaluation and use are also bounded by specific data, speed, participant, and alignment conditions.

N LIMITATIONS

  • 0.624 SRCC on PIPAL and 0.378 on TID2013 mark weaker performance on restoration outputs and global contrast changes, respectively.The contrast result is reported for TID2013.Source: N LIMITATIONS · p. 33
  • The lens ignores displacements in 320 discarded feature directions, which direct optimization against it can exploit; adding the output term sacrifices the triangle inequality.Triangle-inequality violations occurred on at most 107 per million audited triplets; the lens term alone can be used when a strict pseudometric is required.Source: N LIMITATIONS · p. 34
  • On held-out video, JLD-video reaches 0.912 versus 0.932 for VMAF; Figure A18 shows synchronized examples, model choices, and disagreements with MOS.Coral marks disagreement with MOS and visible artifacts.Source: N LIMITATIONS · p. 34, Figure A18 · p. 34
  • The human study has only ten participants, and JLD assumes spatially aligned inputs.Source: N LIMITATIONS · p. 34
  • Full JLD is slower than DISTS; the approximately four-fold speed advantage applies to JLD-fast.Source: N LIMITATIONS · p. 34

Related papers

Loading 2610.05967v1…