Running cross platform Juniper cRPD on Raspberry Pi4

Background

While I have internal access to early ARM versions of cRPD via daily builds, they are actually daily builds, with all their glory and pitfalls. So I thought, wouldn’t it be great to run occasionally a container built for a different platform, even just for the sake of doing a simple and automated unit test as part of CICD pipeline? I think so too ;).

Raspberry Pi 4

I won’t describe the concept of multi platform builds. Please refer to the Docker documentation and buildx on Github: https://github.com/docker/buildx

Installation

Docker needs to be installed and running before executing below steps. Check the latest version of buildx binary on the github page and adjust accordingly. Same goes for the desired cross platform. Replace amd64 with arm64 to try this out on a RaspberryPi.

wget https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm64
mkdir -p ~/.docker/cli-plugins
mv buildx-v0.6.0.linux-arm64 ~/.docker/cli-plugins/docker-buildx
chmod a+rx ~/.docker/cli-plugins/docker-buildx
docker run --privileged --rm tonistiigi/binfmt --install all
$ docker buildx ls

NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS
mybuilder * docker-container
mybuilder0 unix:///var/run/docker.sock running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default docker
default default running linux/amd64, linux/386, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/arm/v7, linux/arm/v6

Deploy “cross platform” cRPD

Load the cRPD image as usual, ignoring doing so for the “wrong” image based on platform it was built for.

$ docker load -i crpd-arm.tgz 

$ docker images|grep crpd|grep 21.3
svl-artifactory.juniper.net/junos-docker-local/crpd/arm64/crpd 21.3I20210427_1631 b44931556479 2 months ago 325MB
crpd 21.2I-20210304_dev_common.0.0857 bf2ad7d82c1e 4 months ago 370MB

$ docker run -ti --privileged --net host --rm --platform arm64 -d svl-artifactory.juniper.net/junos-docker-local/crpd/arm64/crpd:21.3I20210427_1631

$ uname -a

Linux nuc1 5.8.0-63-generic #71~20.04.1-Ubuntu SMP Thu Jul 15 17:46:08 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ docker exec -ti goofy_murdock bash

===>
Containerized Routing Protocols Daemon (CRPD)
Copyright (C) 2020, Juniper Networks, Inc. All rights reserved.
<===

root@nuc1:/# cli
root@nuc1> show version

Hostname: nuc1
Model: cRPD
Junos: 21.3I20210427_1631_bbharaneedar
cRPD package version : 21.3I20210427_1631_bbharaneedar built by bbharaneedar on 2021-04-27 16:41:05 UTC

root@nuc1> show interfaces routing eno1

Interface State Addresses
eno1 Up MPLS enabled
ISO enabled
INET 192.168.0.41
INET6 2a02:168:5f67:0:1e69:7aff:fe0f:e296
INET6 fe80::1e69:7aff:fe0f:e296

root@nuc1> exit

root@nuc1:/# lscpu | grep 'Arch\|name'

Architecture: aarch64
Model name: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz

root@nuc1:/# file /usr/sbin/mgd

/usr/sbin/mgd: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, stripped
root@nuc1:/# exit

This just works! Pay close attention to the output of lscpu, executed with the container shell: aarch64 on an Intel Core ?? Something interesting must be going on here …

Ok. The binaries within that container are clearly compiled for ARM, based on the ‘file’ output. How do these daemons show up on the host itself? Let’s find out:

$ ps ax|grep mgd

1793135 ? Ssl 0:00 /usr/bin/qemu-aarch64 /usr/bin/runsv mgd-api
1793143 ? Ssl 0:00 /usr/bin/qemu-aarch64 /usr/bin/runsv mgd
1793182 ? Sl 0:00 /usr/bin/qemu-aarch64 /usr/sbin/mgd-api -N
1793200 ? Sl 0:00 /usr/bin/qemu-aarch64 /usr/sbin/mgd -N
1793269 ? Sl 0:00 /usr/bin/qemu-aarch64 /usr/sbin/mgd -A -b -T -r -E user "root" logname "root" host "nuc1" agent "JUNOS CLI" current-directory "/etc/sv/xmlproxyd" pid 448 ppid 379
1806496 pts/3 S+ 0:00 grep --color=auto mgd

Every process runs within its own qemu process!

Now how about running amd64 cRPD on ARM, in particular a RaspberryPi (running 64Bit Ubuntu 20.04)? Well, it works equally well:

mwiget@k1:~$ uname -a

Linux k1 5.11.0-1015-raspi #16-Ubuntu SMP PREEMPT Wed Jul 14 11:04:58 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

mwiget@k1:~$ docker ps |grep crpd

1d81264161c1 crpd:21.2R1.10 "/sbin/runit-init.sh" 49 minutes ago Up 49 minutes 22/tcp, 179/tcp, 830/tcp, 3784/tcp, 4784/tcp, 6784/tcp, 7784/tcp, 50051/tcp crpd

mwiget@k1:~$ docker exec -ti crpd bash

===>
Containerized Routing Protocols Daemon (CRPD)
Copyright (C) 2020, Juniper Networks, Inc. All rights reserved.
<===

root@1d81264161c1:/# cli
root@1d81264161c1> show version

Hostname: 1d81264161c1
Model: cRPD
Junos: 21.2R1.10
cRPD package version : 21.2R1.10 built by builder on 2021-06-21 14:13:43 UTC

root@1d81264161c1> show interfaces routing eth0

Interface State Addresses
eth0 Up MPLS enabled
ISO enabled
INET 172.17.0.2
INET6 fe80::42:acff:fe11:2

root@1d81264161c1> exit
root@1d81264161c1:/# lscpu | grep 'Arch\|name'

Architecture: x86_64
Model name: Cortex-A72

root@1d81264161c1:/# file /usr/sbin/mgd

/usr/sbin/mgd: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, stripped

root@1d81264161c1:/# exit
exit
mwiget@k1:~$ ps ax|grep mgd
30466 ? Ssl 0:00 /usr/bin/qemu-x86_64 /usr/bin/runsv mgd-api
30483 ? Ssl 0:00 /usr/bin/qemu-x86_64 /usr/bin/runsv mgd
30518 ? Sl 0:00 /usr/bin/qemu-x86_64 /usr/sbin/mgd-api -N
30522 ? Sl 0:01 /usr/bin/qemu-x86_64 /usr/sbin/mgd -N
32112 ? Zs 0:00 [mgd] <defunct>
32150 pts/1 S+ 0:00 grep --color=auto mgd
mwiget@k1:~$

The cRPD initialization takes a bit longer for an Intel container deployed on RaspberryPi, which leads to delays when trying to launch ‘cli’ right after deployment. It took less than a minute on a RPi.

Conclusion

I’m sure there are pitfalls deploying cross platform containers. But its great to see it works even in net=host mode, hence its very different from running a Virtual Machine, where all the processes are deployed on an emulated kernel (the one within the VM). Here, each process gets a “qemu wrapper” and has access to the host kernel API’s like netlink.

If you want to try this out for yourself and have a Raspberry PI, sign up for the cRPD Free Trial at https://www.juniper.net/us/en/dm/crpd-free-trial.html

Now you know how to deploy the amd64 based container on an arm64 platform.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at WordPress.com.

Up ↑

%d bloggers like this: