Magnum::DebugTools::CompareImage class

Image comparator.

To be used with Corrade::TestSuite. Basic use is really simple:

void ProcessingTest::process() {
    Image2D actual = doProcessing();
    Image2D expected = loadExpectedImage();

    CORRADE_COMPARE_WITH(actual, expected,
        (DebugTools::CompareImage{170.0f, 96.0f}));
}

Based on actual images used, in case of comparison failure the comparator can give for example the following output:

Starting ProcessingTest with 1 test cases...
  FAIL [1] process() at …/debugtools-compareimage.cpp:77
        Images actual and expected have max delta above threshold, actual 189
        but at most 170 expected. Mean delta 13.5776 is within threshold 96.
        Delta image:
          |                                |
          |                                |
          |         ~8070DNMN8$ZD7         |
          |       ?I0:   :++~.  .I0Z       |
          |      7I   ?$D8ZZ0DZ8,  +?      |
          |     ~+   +I        ,7NZZ$      |
          |     :    ~                     |
          |     .    .                     |
          |     ,    :                     |
          |     ~.   +.         +ID8?.     |
          |      ?.  .Z0:     +0I  :7      |
          |      .$$.  ~D8$Z0DZ.  =Z+      |
          |        =8$DI=,. .:+ZDI$        |
          |           :70DNMND$+.          |
          |                                |
          |                                |
        Top 10 out of 66 pixels above max/mean threshold:
          [16,5] #000000ff, expected #fcfcfcff (Δ = 189)
          [16,27] #fbfbfbff, expected #000000ff (Δ = 188.25)
          [15,27] #f2f2f2ff, expected #000000ff (Δ = 181.5)
          [17,5] #000000ff, expected #f1f1f1ff (Δ = 180.75)
          [15,5] #000000ff, expected #efefefff (Δ = 179.25)
          [17,27] #eeeeeeff, expected #000000ff (Δ = 178.5)
          [22,20] #000000ff, expected #e7e7e7ff (Δ = 173.25)
          [18,23] #060606ff, expected #eaeaeaff (Δ = 171)
          [18,9] #e5e5e5ff, expected #040404ff (Δ = 168.75)
          [21,26] #efefefff, expected #0f0f0fff (Δ = 168)
Finished ProcessingTest with 1 errors out of 1 checks.

Supports the following formats:

PixelFormat::RGBA16F and other half-float formats are not supported at the moment. Implementation-specific pixel formats can't be supported.

Supports all PixelStorage parameters. The images don't need to have the same pixel storage parameters, meaning you are able to compare different subimages of a larger image as long as they have the same size.

The comparator first compares both images to have the same pixel format/type combination and size. Each pixel is then first converted to Float vector of corresponding channel count and then the per-pixel delta is calculated as simple sum of per-channel deltas (where $ \boldsymbol{a} $ is the actual pixel value, $ \boldsymbol{e} $ expected pixel value and $ c $ is channel count), with max and mean delta being taken over the whole picture.

\[ \Delta_{\boldsymbol{p}} = \sum\limits_{i=1}^c \dfrac{a_i - e_i}{c} \]

The two parameters passed to the CompareImage(Float, Float) constructor are max and mean delta threshold. If the calculated values are above these threshold, the comparison fails. In case of comparison failure the diagnostic output contains calculated max/meanvalues, delta image visualization and a list of top deltas. The delta image is an ASCII-art representation of the image difference with each block being a maximum of pixel deltas in some area, printed as characters of different perceived brightness. Blocks with delta over the max threshold are colored red, blocks with delta over the mean threshold are colored yellow. The delta list contains X,Y pixel position (with origin at bottom left), actual and expected pixel value and calculated delta.

Sometimes it's desirable to print the delta image even if the comparison passed — for example, to check that the thresholds aren't too high to hide real issues. If the --verbose command-line option is specified, every image comparison with a non-zero delta will print an INFO message in the same form as the error diagnostic shown above.

Special floating-point values

For floating-point input, the comparator treats the values similarly to how Corrade::TestSuite::Comparator<float> behaves for scalars:

  • If both actual and expected channel value are NaN, they are treated as the same (with channel delta being 0).
  • If actual and expected channel value have the same sign of infinity, they are treated the same (with channel delta being 0).
  • Otherwise, the delta is calculated the usual way, with NaN and infinity values getting propagated according to floating-point rules. This means the final per-pixel $ \Delta_{\boldsymbol{p}} $ becomes either NaN or infinity.
  • When calculating the max value, NaN and infinity $ \Delta_{\boldsymbol{p}} $ values are ignored. This is done in order to avoid a single infinity deltas causing all other deltas to be comparatively zero in the ASCII-art representation.
  • The mean value is calculated as usual, meaning that NaN or infinity in $ \Delta_{\boldsymbol{p}} $ "poison" the final value, reliably causing the comparison to fail.

For the ASCII-art representation, NaN and infinity $ \Delta_{\boldsymbol{p}} $ values are always treated as maximum difference.

Comparing against pixel views

For added flexibility, it's possible to use a Corrade::Containers::StridedArrayView2D containing pixel data on the left side of the comparison in both CompareImage and CompareImageToFile. This type is commonly returned from ImageView::pixels() and allows you to do arbitrary operations on the viewed data — for example, comparing pixel data flipped upside down:

CORRADE_COMPARE_WITH(actual.pixels<Color3ub>().flipped<0>(), expected,
    (DebugTools::CompareImage{1.5f, 0.01f}));

For a different scenario, imagine you're comparing data read from a framebuffer to a ground truth image. On many systems, internal framebuffer storage has to be four-component; however your if your ground truth image is just three-component you can cast the pixel data to just a three-component type:

Image2D image = fb.read(fb.viewport(), {PixelFormat::RGBA8Unorm});

CORRADE_COMPARE_AS(Containers::arrayCast<Color3ub>(image.pixels<Color4ub>()),
    "expected.png", DebugTools::CompareImageToFile);

Currently, comparing against pixel views has a few inherent limitations — it has to be cast to one of Magnum scalar or vector types and the format is then autodetected from the passed type, with normalized formats preferred. In practice this means e.g. Math::Vector2<UnsignedByte> will be understood as PixelFormat::RG8Unorm and there's currently no way to interpret it as PixelFormat::RG8UI, for example.

Constructors, destructors, conversion operators

CompareImage(Float maxThreshold, Float meanThreshold) explicit
Constructor.
CompareImage() explicit
Construct with implicit thresholds.

Function documentation

Magnum::DebugTools::CompareImage::CompareImage(Float maxThreshold, Float meanThreshold) explicit

Constructor.

Parameters
maxThreshold Max threshold. If any pixel has delta above this value, this comparison fails
meanThreshold Mean threshold. If mean delta over all pixels is above this value, the comparison fails

Magnum::DebugTools::CompareImage::CompareImage() explicit

Construct with implicit thresholds.

Equivalent to calling CompareImage(Float, Float) with zero values.