Testing with multi-arch Docker images

With the introduction of Docker Buildx CLI plugin it became quite easy to test various platforms using single Dockerfile without any QEMU static builds. In this tutorial we’re going to use a multistage build with the simple C++ program that tests features of various platforms.

First, we need to register QEMU handlers. Please note that the handler registration won’t survive a reboot.

$ docker run --rm --privileged docker/binfmt:a7996909642ee92942dcd6cff44b9b95f08dad64

There is no latest alias for the docker/binfmt and it’s recommended to grab the proper tag on their homepage: https://hub.docker.com/r/docker/binfmt/tags?page=1&ordering=last_updated

The code to detect the platform endianness and its sizes for the basic types:

#include <iostream>

#ifndef PLATFORM
  #define PLATFORM "N/A"
#endif

void platform() {
  std::cout << "Platform: " << PLATFORM << std::endl;
  #ifdef __BYTE_ORDER__
    std::cout << "__BYTE_ORDER__\t\t= " << __BYTE_ORDER__ << " (";
    #if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
      std::cout << "__ORDER_BIG_ENDIAN__";
    #elif __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
      std::cout << "__ORDER_LITTLE_ENDIAN__";
    #else
      std::cout << "unknown";
    #endif
    std::cout << ")" << std::endl;
  #else
    std::cout << "__BYTE_ORDER__ is undefined" << std::endl;
  #endif

  #ifdef __FLOAT_WORD_ORDER__
    std::cout << "__FLOAT_WORD_ORDER__\t= " << __FLOAT_WORD_ORDER__ << " (";
    #if __FLOAT_WORD_ORDER__ == __ORDER_BIG_ENDIAN__
      std::cout << "__ORDER_BIG_ENDIAN__";
    #elif __FLOAT_WORD_ORDER__ == __ORDER_LITTLE_ENDIAN__
      std::cout << "__ORDER_LITTLE_ENDIAN__";
    #else
      std::cout << "unknown";
    #endif
    std::cout << ")" << std::endl;
  #else
    std::cout << "__FLOAT_WORD_ORDER__ is undefined" << std::endl;
  #endif

  std::cout << "sizeof(bool)\t\t= " << sizeof(bool)
    << " (" << 8 * sizeof(bool) << " bits)" << std::endl;
  std::cout << "sizeof(char)\t\t= " << sizeof(char)
    << " (" << 8 * sizeof(char) << " bits)" << std::endl;
  std::cout << "sizeof(wchar_t)\t\t= " << sizeof(wchar_t)
    << " (" << 8 * sizeof(wchar_t) << " bits)" << std::endl;
  std::cout << "sizeof(short)\t\t= " << sizeof(short)
    << " (" << 8 * sizeof(short) << " bits)" << std::endl;
  std::cout << "sizeof(int)\t\t= " << sizeof(int)
    << " (" << 8 * sizeof(int) << " bits)" << std::endl;
  std::cout << "sizeof(long)\t\t= " << sizeof(long)
    << " (" << 8 * sizeof(long) << " bits)" << std::endl;
  std::cout << "sizeof(long long)\t= " << sizeof(long long)
    << " (" << 8 * sizeof(long long) << " bits)" << std::endl;
  std::cout << "sizeof(float)\t\t= " << sizeof(float)
    << " (" << 8 * sizeof(float) << " bits)" << std::endl;
  std::cout << "sizeof(double)\t\t= " << sizeof(double)
    << " (" << 8 * sizeof(double) << " bits)" << std::endl;
  std::cout << "sizeof(void*)\t\t= " << sizeof(void*)
    << " (" << 8 * sizeof(void*) << " bits)" << std::endl;
}

int main() {
  platform();
}

The Dockerfile uses multistage build to compile and run the program:

FROM alpine AS builder
RUN apk add g++
WORKDIR /home
COPY platform.C .

# TARGETPLATFORM is provided by Buildx
ARG TARGETPLATFORM
RUN g++ "-DPLATFORM=\"$TARGETPLATFORM\"" platform.C -o platform -static

FROM alpine
WORKDIR /home
COPY --from=builder /home/platform .
ENTRYPOINT ["./platform"]

buildx does support batch build, but only within the limited output options (like --push). For local testing, I’d prefer to avoid pushing, and rather build the image for the local Docker. Unfortunately, that won’t work in the batch mode, therefore some shell scripting is required.

Let’s prepare all the required files first (within some temporary directory):

$ mkdir multiarch
$ cd multiarch
$ curl -O https://gitlab.com/nuald/nuald-blogspot-com/-/raw/master/site/adoc/platform.C
$ curl -o Dockerfile https://gitlab.com/nuald/nuald-blogspot-com/-/raw/master/site/adoc/multiarch-dockerfile

Get all supported platforms by the buildx builder:

$ PLATFORMS=($(docker buildx inspect | sed -n -e 's/Platforms: \(.*\)/\1/p' | tr ',' "\n"))
$ echo ${PLATFORMS[*]}
linux/amd64 linux/386 linux/arm64 linux/riscv64 linux/ppc64le linux/s390x linux/arm/v7 linux/arm/v6

Build all images (please note that riscv64 platform is not supported by the official Alpine dockerhub):

$ for platform in ${PLATFORMS[@]}; do docker buildx build --platform $platform -o type=docker,name=multiarch/$platform .; done

The list of all official supported Docker images for alternative platforms is available in the documentation: https://github.com/docker-library/official-images#architectures-other-than-amd64

Finally, run all images (you could see that riscv64 image is not available):

$ for platform in ${PLATFORMS[@]}; do docker run -it --rm --platform $platform multiarch/$platform .; done
Platform: linux/amd64
__BYTE_ORDER__		= 1234 (__ORDER_LITTLE_ENDIAN__)
__FLOAT_WORD_ORDER__	= 1234 (__ORDER_LITTLE_ENDIAN__)
sizeof(bool)		= 1 (8 bits)
sizeof(char)		= 1 (8 bits)
sizeof(wchar_t)		= 4 (32 bits)
sizeof(short)		= 2 (16 bits)
sizeof(int)		= 4 (32 bits)
sizeof(long)		= 8 (64 bits)
sizeof(long long)	= 8 (64 bits)
sizeof(float)		= 4 (32 bits)
sizeof(double)		= 8 (64 bits)
sizeof(void*)		= 8 (64 bits)
Platform: linux/386
__BYTE_ORDER__		= 1234 (__ORDER_LITTLE_ENDIAN__)
__FLOAT_WORD_ORDER__	= 1234 (__ORDER_LITTLE_ENDIAN__)
sizeof(bool)		= 1 (8 bits)
sizeof(char)		= 1 (8 bits)
sizeof(wchar_t)		= 4 (32 bits)
sizeof(short)		= 2 (16 bits)
sizeof(int)		= 4 (32 bits)
sizeof(long)		= 4 (32 bits)
sizeof(long long)	= 8 (64 bits)
sizeof(float)		= 4 (32 bits)
sizeof(double)		= 8 (64 bits)
sizeof(void*)		= 4 (32 bits)
Platform: linux/arm64
__BYTE_ORDER__		= 1234 (__ORDER_LITTLE_ENDIAN__)
__FLOAT_WORD_ORDER__	= 1234 (__ORDER_LITTLE_ENDIAN__)
sizeof(bool)		= 1 (8 bits)
sizeof(char)		= 1 (8 bits)
sizeof(wchar_t)		= 4 (32 bits)
sizeof(short)		= 2 (16 bits)
sizeof(int)		= 4 (32 bits)
sizeof(long)		= 8 (64 bits)
sizeof(long long)	= 8 (64 bits)
sizeof(float)		= 4 (32 bits)
sizeof(double)		= 8 (64 bits)
sizeof(void*)		= 8 (64 bits)
Unable to find image 'multiarch/linux/riscv64:latest' locally
docker: Error response from daemon: pull access denied for multiarch/linux/riscv64, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.
Platform: linux/ppc64le
__BYTE_ORDER__		= 1234 (__ORDER_LITTLE_ENDIAN__)
__FLOAT_WORD_ORDER__	= 1234 (__ORDER_LITTLE_ENDIAN__)
sizeof(bool)		= 1 (8 bits)
sizeof(char)		= 1 (8 bits)
sizeof(wchar_t)		= 4 (32 bits)
sizeof(short)		= 2 (16 bits)
sizeof(int)		= 4 (32 bits)
sizeof(long)		= 8 (64 bits)
sizeof(long long)	= 8 (64 bits)
sizeof(float)		= 4 (32 bits)
sizeof(double)		= 8 (64 bits)
sizeof(void*)		= 8 (64 bits)
Platform: linux/s390x
__BYTE_ORDER__		= 4321 (__ORDER_BIG_ENDIAN__)
__FLOAT_WORD_ORDER__	= 4321 (__ORDER_BIG_ENDIAN__)
sizeof(bool)		= 1 (8 bits)
sizeof(char)		= 1 (8 bits)
sizeof(wchar_t)		= 4 (32 bits)
sizeof(short)		= 2 (16 bits)
sizeof(int)		= 4 (32 bits)
sizeof(long)		= 8 (64 bits)
sizeof(long long)	= 8 (64 bits)
sizeof(float)		= 4 (32 bits)
sizeof(double)		= 8 (64 bits)
sizeof(void*)		= 8 (64 bits)
Platform: linux/arm/v7
__BYTE_ORDER__		= 1234 (__ORDER_LITTLE_ENDIAN__)
__FLOAT_WORD_ORDER__	= 1234 (__ORDER_LITTLE_ENDIAN__)
sizeof(bool)		= 1 (8 bits)
sizeof(char)		= 1 (8 bits)
sizeof(wchar_t)		= 4 (32 bits)
sizeof(short)		= 2 (16 bits)
sizeof(int)		= 4 (32 bits)
sizeof(long)		= 4 (32 bits)
sizeof(long long)	= 8 (64 bits)
sizeof(float)		= 4 (32 bits)
sizeof(double)		= 8 (64 bits)
sizeof(void*)		= 4 (32 bits)
Platform: linux/arm/v6
__BYTE_ORDER__		= 1234 (__ORDER_LITTLE_ENDIAN__)
__FLOAT_WORD_ORDER__	= 1234 (__ORDER_LITTLE_ENDIAN__)
sizeof(bool)		= 1 (8 bits)
sizeof(char)		= 1 (8 bits)
sizeof(wchar_t)		= 4 (32 bits)
sizeof(short)		= 2 (16 bits)
sizeof(int)		= 4 (32 bits)
sizeof(long)		= 4 (32 bits)
sizeof(long long)	= 8 (64 bits)
sizeof(float)		= 4 (32 bits)
sizeof(double)		= 8 (64 bits)
sizeof(void*)		= 4 (32 bits)

And surely, it all could be properly automated and integrated with the required CI/CD systems.

Comments

Post a Comment

Popular posts from this blog

Web application framework comparison by memory consumption

Memory consumption is slightly specific to my area of software development now, but I did some research recently and maybe these results can be useful for others. Of course, I know that precious comparison is very difficult to carry out, but actually I needed only overall picture. And let me admit that results are pretty interesting and even frustrated (at least for me). As a basis I took so-starving project, and measured initial RSS (Resident Set Size) of the each process (local development webservers). Platform: x86_64 Linux (latest Ubuntu with all updates). As a reference, here is the RSS of the interpreters in interactive console mode: Interpreter Version RSS (kB) stackless python 2.6.4 3916 ruby (via irb) 1.8.7 4664 python 2.7.1 5624 php 5.3.5 6924 v8 (via node.js shell) 2.5.9.9 8796 One more reference - the most simplest WSGI app ( example  in the Python documentation). It's RSS: 7336 Kb , so I assume it's almost impossible to consume l
Read more

Trac Ticket Workflow

Trac is the wiki and issue tracking system that is the most used for my software development projects. Let me share few hints about its ticket workflow, especially about the statuses, the roles of the participants, and let me give an example of the workflow config. But if you are interested in other things like Trac plugins, source browser, customizing, etc, let me know. All the information given below regards the workflow described at the end of the article. The original Trac workflow is missed vital features like the support for ticket verifying, or the required ticket statuses range, so I do not use it in my projects. The description of the main statuses: new - the ticket has been created; in-progress - the developer has began to work with the ticket; fixed - the ticket is fixed (the problem issued in the ticket is solved); verified - the ticket is verified (tester or PM has verified that the ticket is fixed correctly); closed - the ticket is closed (it's not
Read more

Shellcode detection using libemu

Shellcode can be seen as a list of instructions that has been developed in a manner that allows it to be injected in an application during runtime. Each security researcher face the shellcodes during their work, and in this article I'll show how to detect shellcodes using Python (via libemu Python binding). Few words about libemu : libemu is a small library written in C offering basic x86 emulation and shellcode detection using GetPC heuristics. Intended use is within network intrusion/prevention detections and honeypots. The information on the site is not actual in some places, so I'll give direct and clear instruction how to get and install libemu. Clone the git repository: $ git clone git://git.carnivore.it/libemu.git Firstly, configure, make and install libemu itself (without binding): $ autoreconf -v -i $ ./configure --prefix=/opt/libemu $ make $ sudo make install If you set up prefix as shown above, you have to add the library path to /etc/ld.so.conf file
Read more