Visualizing images from VK-GL-CTS test results
When running OpenGL or Vulkan tests normally from VK-GL-CTS, the test suite executable will usually produce a file named TestResults.qpa containing test results in XML format. Sometimes either by default or when a test fails, this file contains output images obtained typically from the test output itself and maybe a reference image the former is being compared to. In addition, sometimes an error mask is provided so implementors can easily detect cases of pixels falling outside the expected result range.
These images are normally converted to PNGs using 32-bits per pixel (RGBA8) and represented in the test log as a string of Base64-encoded binary data. Representing the result image in that format can be an approximation exercise when the original format or type is very different (like R32G32B32A32_SFLOAT or a 3D image), but in some situations the result image can be faithfully represented in that chunk of PNG-encoded data.
To view those images, the VK-GL-CTS README file mentioned the possibility of
using the
Cherry tool.
Given that it requires setting up a web server and running tests in a special
way, some people would typically rely on external tools or scripts instead,
like the base64
command line tool included in GNU coreutils, which can encode
and decode Base64 content.
My Igalia colleague Eduardo Lima, however, had
another idea. Most people have a tool in their systems which is capable of
directly displaying Base64-encoded PNG data: a web browser. With a little bit
of Javascript, he had created a self-contained and single-page web app you
could use to view images by pasting the PNG-encoded version of those images
directly. The tool created <img>
elements on the page from the pasted text,
and used data:image/png;base64,<Base64-encoded data>
as the image src
property. It also allowed comparing two images side by side and was, in
general, incredibly handy.
Alejandro PiƱeiro, currently working on the Raspberry Pi Vulkan driver also at Igalia, suggested improving the existing tool by allowing it to read TestResults.qpa files directly in order to reduce friction. I’m far from a web programmer and I’m sorry for the many times I have probably sinned along the way, but I took his suggestions and my own pet peeves with the existing tool and implemented an improved version of it. I submitted my new tool for review and inclusion as part of VK-GL-CTS and I’m happy to say it landed not long ago. If you use VK-GL-CTS, do not hesitate to open qpa_image_viewer.html from the scripts subdirectory in your web browser and give it a go when visualizing test results.
I have also uploaded a copy of
the tool to my own personal space at Igalia. Feel free to bookmark it and use
it when needed. It’s just a few KB in size. As I mentioned before, it’s
self-contained and standalone, so everything happens locally in your browser.
You can read its source code as it’s also relatively small. You can open
TestResults.qpa files directly from your system and you can also paste chunks
of text containing <Image>
elements. It accumulates images in the images
section at the bottom, i.e., every time you tell it to process a chunk of text
or process a new file, it will add the images it finds to the images section.
To ease identifying problems, it also includes a built-in zoom tool. Images
have the image-rendering: pixelated
CSS property, which is supported in
Chromium and WebKit-based browsers (sadly, not Firefox for now), making the
zooming process use the nearest-neighbor approximation instead of a linear
interpolation, so pixels are represented as faithfully as possible when
scaling.