Using Docker buildx for native multiarch builds.

Posted on Dec 24, 2022

TL;DR

I wanted to build multiarch images using multiple machines with different architectures instead of QEMU.

# add some amd64 machine that has docker
docker context create bob --docker "host=ssh://bob@10.0.0.2"
# create a buildx builder with your local machine added to it
docker buildx create \
    --name multiarch --node arm64 \
    --platform linux/arm64/v8 \
    default
# add the amd64 machine to it
docker buildx create --append \
    --name multiarch --node amd64 \
    --platform linux/amd64,linux/amd64/v2,linux/386\
    bob
# build & push your image
docker buildx build \
    --builder multiarch \
    --platform linux/amd64,linux/arm64/v8 \
    --push --tag username/imagename .

# the amd64 image will be build natively on the amd64 machine,
# the arm64 machine will be build natively on the arm64 machine

Actual post

I wanted to build a multiarch image for linux/arm64/v8 and linux/amd64, however the x86 builds were really slow when running through QEMU on my Apple Silicon laptop. Turns out, you can tell Docker to use another machine (which happens to be x86_64), to build the linux/amd64 images, and use the local Docker Desktop instance for the ARM stuff.

Here’s how:

Set up Docker on a machine (or many machines) that supports the architecture(s) you want to build for. Make sure you can SSH into it without using passwords (use SSH keys, and maybe ssh-agent if keys don’t work on their own).

Also, the user you’re SSH-ing into needs to be able to access Docker.
Create a Docker context for the remote machine.
```
docker context create <context_name> --docker "host=ssh://<USER>@<HOST>:<PORT>"
```
<USER>@ can be omitted - your local username will be used. :<PORT> can also be omitted if it’s 22.

Let’s say we have an amd64 host on ip 10.0.0.2, and we want to call it bob (the builder).
```
docker context create bob --docker "host=ssh://bob@10.0.0.2"
```
Create a buildx builder for your local computer.

In this case, my local machine supports linux/arm64/v8 natively, but it also can build amd64 images through QEMU, so I explicitly set --platform to just linux/arm64/v8. I might also add linux/arm/v7 for images meant to run on something like a Raspberry Pi here, because even though it will still run via QEMU, I don’t have anything better to build for armv7 anyway. (it’s comma separated)
```
docker buildx create \
    --name multiarch --node arm64 \
    --platform linux/arm64/v8 \
    default
```
multiarch will be the name of our builder, --node is the name for the node we’re adding to the builder (it doesn’t have to contain the architecture, it can be anything, only the --platform value is taken into consideration by Docker).

default is the name of the default context, which usually is your local machine. If not, replace it with whatever you use.
Now, the fun part - add additional nodes for other architectures! In our example, we want to add bob which can build linux/amd64 images natively.
```
docker buildx create --append \
    --name multiarch --node amd64 bob
```
Notice the --append - it tells Docker to add the node to an existing builder.

I’ve omitted --platform here, because in my case, bob only supports x86 anyways, so I don’t have to limit it artificially. This was different for the local node, as Docker Desktop on MacOS comes with cross-platform build support preconfigured.
Set the builder as default (optional, you can use --builder multiarch for docker buildx build to set the builder manually)
```
docker buildx use multiarch
```
Finally, you can build your images:
```
docker buildx build \
    --platform linux/amd64,linux/arm64/v8 \
    --push --tag username/imagename .
```
Unfortunately, it’s not yet possible to load multiarch images into your local Docker daemon, which means you’re gonna have to tag and --push to the Docker registry.

You can still use --load instead of --push (to load into your local daemon) if you only specify a single --platform. Idk, might be useful for testing or something.

Enjoy.