Macvlan Docker interface with explicit endpoint MAC address

Probably one of thos “unicorn” problems, but I was in need of running a Juniper vMX Docker Container attached to a network interface using a statically assigned IP and MAC address. The use case is to run vMX on a baremetal server in a public cloud.

Macvlan is ideal to share a single network interface with a compute host and containers on it. But it currently doesn’t allow to specify the MAC address, except when used as the single container interface: See [#1009](https://github.com/docker/libnetwork/issues/1009).

I need to define the MAC address, which is provided by the public cloud provider. But how? docker network connect doesn’t have ‘–mac-address’ option (docker run has it, but only for the default/first network interface).

So, time to play with macvlan interfaces and network namespaces myself ..

For documentation purpose, lets say the baremetal server got the following IP addresses assigned to interface enp2s0 (using IPv4 example addresses from RFC 5737:

203.0.113.10/24
203.0.113.22/24 with MAC address 00:50:56:00:1A:E2

After launching the vMX container with the default network eth0 attached to docker0, it will wait for additional interfaces to be attached to the running container. This is where I execute following script, specifing the container name (vmx1), physical network interface to attach to (enp2s0) and the MAC address to assign to the macvlan based interface in the container. The script requires sudo privileges.

$ ./add-macvlan-link.sh vmx1 enp2s0 00:50:56:00:1A:E2
CONTAINER=vmx1 INTERFACE=enp2s0 MACADDR=00:50:56:00:1A:E2
vmx1 has pid 1815
creating namespace for container vmx1
vmx1 has 1 eth interfaces
eth1 attached to container vmx1 using 00:50:56:00:1A:E2

And the script itself:

#!/bin/bash
CONTAINER=$1
INTERFACE=$2
MACADDR=$3
if [ -z "$MACADDR" ]; then
    echo "$0 <container> <physical interface> <mac-address>"
    exit 1
fi

echo "CONTAINER=$CONTAINER INTERFACE=$INTERFACE MACADDR=$MACADDR"

sudo mkdir -p /var/run/netns

pid=$(docker inspect -f "{{.State.Pid}}" $CONTAINER)
if [ -z "$pid" ]; then
    echo "Can't find pid for container $CONTAINER"
    exit 1
fi

echo "$CONTAINER has pid $pid"
echo "creating namespace for container $CONTAINER"
sudo ln -sf /proc/$pid/ns/net /var/run/netns/$CONTAINER

ifcount=$(sudo ip netns exec $CONTAINER ifconfig -a | grep ^eth | wc -l)

echo "$CONTAINER has $ifcount eth interfaces"

sudo ip link add v${CONTAINER}-${ifcount} link $INTERFACE type macvlan mode bridge
sudo ip link set v${CONTAINER}-${ifcount} address $MACADDR
sudo ip link set v${CONTAINER}-${ifcount} name eth${ifcount} netns $CONTAINER
sudo ip netns exec $CONTAINER ifconfig eth${ifcount} up

echo "eth${ifcount} attached to container $CONTAINER using $MACADDR"

The script first finds out the process Id of the running container, then creates a symlink under /var/run/netns in order for ip netns commands to work, then creates a an IP link of type macvlan in bridge mode using a unique name (combining container name and unused ethX number) and places it into the network namespace of the container. Finally, the interface is brought up.
Typically one would also set the MTU, but because this is a public cloud, I have to leave it at its default of 1500.

Once the vMX is running, I configured interface ge–0/0/0 with the IPv4 address and add a static route and voila. Success!

I can log into the vMX over the Internet:

$ ssh 203.0.113.111
Last login: Mon Oct  1 14:51:35 2018 from 212.51.142.77
--- JUNOS 18.3R1.9 Kernel 64-bit  JNPR-11.0-20180816.8630ec5_buil
mwiget@vmx1>
mwiget@vmx1> show interfaces terse ge-0/0/0
Interface               Admin Link Proto    Local                 Remote
ge-0/0/0                up    up
ge-0/0/0.0              up    up   inet     203.0.113.111/24
                                   multiservice

FYI, it is not possible to reach the vMX container via IP from the compute node itself. You can find out more about macvlan here: Use macvlan networks.