Magnum::DebugTools::CompareImage class
#include <Magnum/DebugTools/CompareImage.h>

Image comparator for Corrade::TestSuite.

The simplest way to use the comparator is by passing it to CORRADE_COMPARE_AS() along with an actual and expected image, as shown below. That will compare image sizes, pixel formats and pixel data for equality taking into account pixel storage parameters of each image, without requiring you to manually go through all relevant properties and looping over pixel data:

Image2D actual{…}, expected{…};
CORRADE_COMPARE_AS(actual, expected, DebugTools::CompareImage);

Where the comparator actually shines, however, is comparing with some delta — since images produced by real-world hardware, algorithms and lossy compression schemes are rarely exactly bit-equal. Using CORRADE_COMPARE_WITH() instead to be able to pass parameters to the constructor, it takes a max delta for a single pixel and mean delta averaged over all pixels:

CORRADE_COMPARE_WITH(actual, expected, (DebugTools::CompareImage{170.0f, 96.0f}));

Based on actual images used, in case of a comparison failure the comparator can give for example the following output:

Starting ProcessingTest with 1 test cases...
  FAIL [1] process() at …/debugtools-compareimage.cpp:77
        Images actual and expected have max delta above threshold, actual 189 but at most 170 expected. Mean delta 13.5776 is within threshold 96. Delta image:
          |                                |
          |                                |
          |         ~8070DNMN8$ZD7         |
          |       ?I0:   :++~.  .I0Z       |
          |      7I   ?$D8ZZ0DZ8,  +?      |
          |     ~+   +I        ,7NZZ$      |
          |     :    ~                     |
          |     .    .                     |
          |     ,    :                     |
          |     ~.   +.         +ID8?.     |
          |      ?.  .Z0:     +0I  :7      |
          |      .$$.  ~D8$Z0DZ.  =Z+      |
          |        =8$DI=,. .:+ZDI$        |
          |           :70DNMND$+.          |
          |                                |
          |                                |
        Top 10 out of 66 pixels above max/mean threshold:
          [16,5] #000000ff, expected #fcfcfcff (Δ = 189)
          [16,27] #fbfbfbff, expected #000000ff (Δ = 188.25)
          [15,27] #f2f2f2ff, expected #000000ff (Δ = 181.5)
          [17,5] #000000ff, expected #f1f1f1ff (Δ = 180.75)
          [15,5] #000000ff, expected #efefefff (Δ = 179.25)
          [17,27] #eeeeeeff, expected #000000ff (Δ = 178.5)
          [22,20] #000000ff, expected #e7e7e7ff (Δ = 173.25)
          [18,23] #060606ff, expected #eaeaeaff (Δ = 171)
          [18,9] #e5e5e5ff, expected #040404ff (Δ = 168.75)
          [21,26] #efefefff, expected #0f0f0fff (Δ = 168)
Finished ProcessingTest with 1 errors out of 1 checks.

Supports the following formats:

Packed depth/stencil formats are not supported at the moment, however you can work around that by making separate depth/stencil pixel views and comparing the views to a depth/stencil-only ground truth images. Implementation-specific pixel formats can't be supported.

Supports all PixelStorage parameters. The images don't need to have the same pixel storage parameters, meaning you are able to compare different subimages of a larger image as long as they have the same size.

The comparator first compares both images to have the same pixel format/type combination and size. Each pixel is then first converted to Float vector of corresponding channel count and then the per-pixel delta is calculated as simple sum of per-channel deltas (where $ \boldsymbol{a} $ is the actual pixel value, $ \boldsymbol{e} $ expected pixel value and $ c $ is channel count), with max and mean delta being taken over the whole picture.

\[ \Delta_{\boldsymbol{p}} = \sum\limits_{i=1}^c \dfrac{a_i - e_i}{c} \]

The two parameters passed to the CompareImage(Float, Float) constructor are max and mean delta threshold. If the calculated values are above these threshold, the comparison fails. In case of comparison failure the diagnostic output contains calculated max/meanvalues, delta image visualization and a list of top deltas. The delta image is an ASCII-art representation of the image difference with each block being a maximum of pixel deltas in some area, printed as characters of different perceived brightness. Blocks with delta over the max threshold are colored red, blocks with delta over the mean threshold are colored yellow. The delta list contains X,Y pixel position (with origin at bottom left), actual and expected pixel value and calculated delta.

Sometimes it's desirable to print the delta image even if the comparison passed — for example, to check that the thresholds aren't too high to hide real issues. If the --verbose command-line option is specified, every image comparison with a non-zero delta will print an INFO message in the same form as the error diagnostic shown above.

Special floating-point values

For floating-point input, the comparator treats the values similarly to how Corrade::TestSuite::Comparator<float> behaves for scalars:

  • If both actual and expected channel value are NaN, they are treated as the same (with channel delta being 0).
  • If actual and expected channel value have the same sign of infinity, they are treated the same (with channel delta being 0).
  • Otherwise, the delta is calculated the usual way, with NaN and infinity values getting propagated according to floating-point rules. This means the final per-pixel $ \Delta_{\boldsymbol{p}} $ becomes either NaN or infinity.
  • When calculating the max value, NaN and infinity $ \Delta_{\boldsymbol{p}} $ values are ignored. This is done in order to avoid a single infinity deltas causing all other deltas to be comparatively zero in the ASCII-art representation.
  • The mean value is calculated as usual, meaning that NaN or infinity in $ \Delta_{\boldsymbol{p}} $ "poison" the final value, reliably causing the comparison to fail.

For the ASCII-art representation, NaN and infinity $ \Delta_{\boldsymbol{p}} $ values are always treated as maximum difference.

Comparing against pixel views

For added flexibility, it's possible to use a Corrade::Containers::StridedArrayView2D containing pixel data on the left side of the comparison in both CompareImage and CompareImageToFile. This type is commonly returned from ImageView::pixels() and allows you to do arbitrary operations on the viewed data — for example, comparing pixel data flipped upside down:

CORRADE_COMPARE_WITH(actual.pixels<Color3ub>().flipped<0>(), expected,
    (DebugTools::CompareImage{1.5f, 0.01f}));

For a different scenario, imagine you're comparing data read from a framebuffer to a ground truth image. On many systems, internal framebuffer storage has to be four-component; however your if your ground truth image is just three-component you can cast the pixel data to just a three-component type:

Image2D image = fb.read(fb.viewport(), {PixelFormat::RGBA8Unorm});

CORRADE_COMPARE_AS(Containers::arrayCast<Color3ub>(image.pixels<Color4ub>()),
    "expected.png", DebugTools::CompareImageToFile);

The pixel views are expected to be cast to one of Magnum scalar or vector types. The format is then autodetected from the passed type. For types that map to more than one PixelFormat, such as Vector3ub that can be PixelFormat::RGB8Unorm, RGB8Srgb or RGB8UI, an attempt is made to match the pixel format of the expected image if possible. If not possible, such as comparing a Vector3ub view to a PixelFormat::RGB8Snorm, the comparison fails the same way as if comparing two views of different pixel formats. Byte and short color types always autodetect to normalized (sRGB) pixel formats, never to integer formats.

Constructors, destructors, conversion operators

CompareImage(Float maxThreshold, Float meanThreshold) explicit
Constructor.
CompareImage() explicit
Construct with implicit thresholds.

Function documentation

Magnum::DebugTools::CompareImage::CompareImage(Float maxThreshold, Float meanThreshold) explicit

Constructor.

Parameters
maxThreshold Max threshold. If any pixel has delta above this value, this comparison fails
meanThreshold Mean threshold. If mean delta over all pixels is above this value, the comparison fails

Magnum::DebugTools::CompareImage::CompareImage() explicit

Construct with implicit thresholds.

Equivalent to calling CompareImage(Float, Float) with zero values.