Reproducible OpenGL

I recently learned the hard way that OpenGL is not a “pixel exact” specification. This is coming straight from the horse’s mouth (Appendix A of the OpenGLR Graphics System: A Specification (Version 4.4 (Core Profile) - March 19, 2014))

The OpenGL specification is not pixel exact. It therefore does not guarantee an exact match between images produced by different GL implementations.

Although the result is deterministic (a given machine always returns the same result), it usually isn’t reproducible on different hardware and/or driver. This means that your colleagues who use the exact same data and the exact same code to produce a figure will get a slightly different image. Of course, in many cases it doesn’t matter because monitors and printers are usually not properly calibrated anyway.

However, it makes reliable testing of visualization libraries harder. For example, the plotly.js library renders hundreds of figures which are compared pixel by pixel to baseline images in order to guarantee consistency and correctness across releases. This ensures that those figures will always look the same or at least won’t be altered without developers noticing.

Figure 1. Example plotly.js 3D figure generated from gl3d_ibm-plot.json

Software rendering

One solution is to not rely on the black-box magic that is hardware acceleration and perform the rendering in software. Software defined visualization as Intel puts it allows working with datasets when GPU hardware isn’t available or is limiting. It runs on anything from laptops to workstations to compute nodes in supercomputers.

But is it reproducible? In my tests, the newer more performing llvmpipe and openswr were not but softpipe was reproducible! 🎉🎉🎉

Docker container

Setting up MESA on Ubuntu 18.04 (Bionic Beaver) under Docker:

FROM ubuntu:bionic

RUN apt-get update && apt-get install -y libtool-bin autoconf python-pip libx11-dev libxext-dev x11proto-core-dev x11proto-gl-dev libglew-dev freeglut3-dev bison flex

# Build Mesa
RUN apt-get install -y wget pkg-config zlib1g-dev llvm-dev
RUN wget https://mesa.freedesktop.org/archive/mesa-18.2.4.tar.xz
RUN tar xf mesa-18.2.4.tar.xz
RUN mkdir mesa-18.2.4/build
WORKDIR mesa-18.2.4/build

RUN ../configure --disable-dri \
               --disable-egl \
               --disable-gbm \
               --with-gallium-drivers=swrast,swr \
               --with-platforms=x11 \
               --prefix=/usr/local/ \
               --enable-gallium-osmesa \
               --disable-xvmc --disable-vdpau --disable-va \
               --with-swr-archs=avx

RUN make -j `numprocs` && make install

# Setup our environment variables.
ENV XVFB_WHD="1920x1080x24"\
    DISPLAY=":99" \
    LIBGL_ALWAYS_SOFTWARE="1" \
    GALLIUM_DRIVER="softpipe"

Creating a baseline image

Using this Docker container running Orca, we can now render reproducibly any plotly.js figure on different machines! The result is pixel perfect but be patient: everything is rendered on the CPU!

plotly.js mock 14.json
Figure 2. Example plotly.js baseline image generated from gl3d_ibm-plot.json using reproducible Orca

References:

  • software
  • opengl
  • plotly