Junos BNG Pseudowire Headend Termination (PWHT) demo

Junos BNG Pseudowire Headend Termination (PWHT) demo

Maybe it is just me, but I often find myself googling for working configuration examples, followed by searching thru my own files. Well, I was in need of an end-to-end base configuration that terminates DHCP subscriber sessions coming into the BNG over q-in-q within a pseudowire.

Before I end up losing my working example again, I decided to place a working demo on gitlab: https://gitlab.com/mwiget/bng-pwht-demo and write this brief blog post.

This vMX Docker Container based demo shows very basic, DHCP based, subscriber sessions over Q-in-Q, once directly attached to the BNG and once over a l2circuit, terminating directly on the BNG. This is called pseudwire headend termination and documented in the Juniper TechLibrary: Pseudowire Subscriber Logical Interfaces Overview.

A more detailed example configuration can be found in the Network Configuration Example: Configuring the Broadband Edge as a Service Node Within Seamless MPLS Network Designs document.

Network Topology

Network Toplogy
Network Toplogy

Containers dhcptester1 and dhcptester2 are simulation multiple DHCP clients each with double tagged VLANs, with one subscriber per inner VLAN (aka CVLAN model).

PE & BNG are vmx-docker-light containers, running vMX 18.2R1in light mode. The Junos configurations for PE and BNG can be found in bng.conf and pe.conf.

The whole setup can be brought to live by running make, which launches docker-compose.yml.



Clone this repo:

git clone https://gitlab.com/mwiget/bng-pwht
cd bng-pwht

Download the KVM version of vMX, e.g. vmx-bundle–18.2R1.9.tgz, and extract the QCOW2 image junos-vmx-x86–64–18.2R1.9.qcow2, then remove the other files:

tar zxf vmx-bundle-18.2R1.9.tgz
mv vmx/images/junos-vmx-x86-64-18.2R1.9.qcow2 .
rm -rf vmx

The qcow2 image is referenced in docker-compose.yml. Update accordingly, if you use a different Junos version.

Make sure you have at least 2048MB of hugepages configured on your baremetal linux server:

$ cat /proc/meminfo |grep Huge
AnonHugePages:   2093056 kB
ShmemHugePages:        0 kB
HugePages_Total:      16
HugePages_Free:       14
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:    1048576 kB

The examples show 14 pages of 1GB free, 2 are in use (by the 2 vMX instances PE and BNG). Edit /etc/default/grub and add hugepages, then activate them by running , followed by a reboot:

$ grep hugep /etc/default/grub
GRUB_CMDLINE_LINUX="default_hugepagesz=1G hugepagesz=1G hugepages=16"
$ sudo update-grub
$ sudo reboot

Launch the demo

Launch all containers via Makefile. Besides using docker-compose to bring up all the containers in the background, it will also download the vMX evaluation license key and copy the users (yours) public ssh keyfile from $HOME/.ssh/ directory and create a matching user account in the BNG and PE vMX routers:

$ make up
docker-compose up -d
Creating network "bngpwht_net1-mgmt" with the default driver
Creating network "bngpwht_net2-subs" with the default driver
Creating network "bngpwht_net5-core" with the default driver
Creating pe ... done
Creating bng ... done
Creating dhcptester1 ... done
Creating pe ...
Creating dhcptester2 ...
Creating dhcptester1 ...

If this is first time you run the ‘make up’, it will also download the containers from docker hub.

Use ‘docker ps’ to verify the containers have successfully started:

$ docker ps
CONTAINER ID        IMAGE                                 COMMAND                  CREATED              STATUS              PORTS                                           NAMES
73b8bcbd6336        marcelwiget/dhcptester:latest         "/usr/bin/dhcptester…"   About a minute ago   Up 56 seconds                                                       dhcptester1
9267596b0850        marcelwiget/dhcptester:latest         "/usr/bin/dhcptester…"   About a minute ago   Up About a minute                                                   dhcptester2
166e7a5beea4        marcelwiget/vmx-docker-light:latest   "/launch.sh"             About a minute ago   Up About a minute>22/tcp,>830/tcp   bng
4d7b9e8ab88d        marcelwiget/vmx-docker-light:latest   "/launch.sh"             About a minute ago   Up About a minute>22/tcp,>830/tcp   pe

If any of the containers isn’t running, check its log file for possible root causes:

$ docker logs bng

BNG and PE will take a few minutes to boot up. You can watch its progress by continously monitoring their log files via:

$ docker logs -f bng
. . .
starting mpcsd
grep: /etc/riot/runtime.conf: No such file or directory
fpc.core.push.sh: no process found
mpc :
tnp_hello_tx: no process found
cat: /var/jnx/card/local/type: No such file or directory
tx_hello_tx: Failed to get card type defaulting to 0
cat: /var/jnx/card/local/slot: No such file or directory
tx_hello_tx: Failed to get card slot defaulting to 0
tnp_hello_tx: Board type 0
tnp_hello_tx: Board slot 0
Setting Up DPDK for Docker env
cat: /var/jnx/card/local/type: No such file or directory
grep: /etc/riot/init.conf: No such file or directory
config file /etc/riot/init.conf
config file /etc/riot/runtime.conf
Disabled QOS
Disabled AlternatePriorityMode
Unable to open config file /etc/riot/shadow
cat: /etc/vmxt/init.conf: No such file or directory
cat: /boot/loader.conf: No such file or directory
cat: /etc/vmxt/init.conf: No such file or directory
cat: /boot/loader.conf: No such file or directory
cat: /etc/vmxt/init.conf: No such file or directory
cat: /boot/loader.conf: No such file or directory
cat: /etc/vmxt/init.conf: No such file or directory
cat: /boot/loader.conf: No such file or directory

Once you see above messages about RIOT, you are all set. Ignore the ‘No such file or directory’ warnings.

Find out the assigned fxp0 IP addresses for BNG and PE:

$ make ps
docker-compose ps
   Name                  Command               State                  Ports
bng           /launch.sh                       Up>22/tcp,
dhcptester1   /usr/bin/dhcptester -s 1 - ...   Up
dhcptester2   /usr/bin/dhcptester -e 2 - ...   Up
pe            /launch.sh                       Up>22/tcp,
vMX bng ( ) root password to aezoozooteeweenohteingoh
vMX pe ( ) root password to razacaeshoungiphahwejohb

The last 2 lines show the IP addresses and their root passwords (only valid when logging in via the serial console by launching ‘docker attach [bng|pe]’).

Log into the BNG and list the subscriber summary. If you see 10 DHCP sessions, then everything worked find. Each dhcptester instance is configured to simulate 5 clients. This number can be increased into thousands and more by editing the parameter ‘-n ’ in the launch option in the docker-compose.yml file:

$ ssh
The authenticity of host ' (' can't be established.
ECDSA key fingerprint is SHA256:txxFHA7YHpeK5Eaarx+F6I2tZ9doDqpSQ4zrcR2lUnw.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.
--- JUNOS 18.2R1.9 Kernel 64-bit  JNPR-11.0-20180614.6c3f819_buil
mwiget@bng> show subscribers summary

Subscribers by State
   Active: 20
   Total: 20

Subscribers by Client Type
   DHCP: 10
   VLAN: 10
   Total: 20


List the detailed subscriber view:

mwiget@bng> show subscribers
Interface             IP Address/VLAN ID                      User Name                      LS:RI
ge-0/0/0.3221225472   0x8100.1 0x8100.2                                                 default:default
ge-0/0/0.3221225473   0x8100.1 0x8100.3                                                 default:default
ge-0/0/0.3221225474   0x8100.1 0x8100.4                                                 default:default
ge-0/0/0.3221225475   0x8100.1 0x8100.5                                                 default:default
ge-0/0/0.3221225476   0x8100.1 0x8100.1                                                 default:default
ge-0/0/0.3221225473                                                        default:default
ge-0/0/0.3221225474                                                        default:default
ge-0/0/0.3221225475                                                        default:default
ge-0/0/0.3221225476                                                        default:default
ge-0/0/0.3221225472                                                        default:default
ps0.3221225483        0x8100.2 0x8100.3                                                 default:default
ps0.3221225483                                                        default:default
ps0.3221225484        0x8100.2 0x8100.4                                                 default:default
ps0.3221225484                                                        default:default
ps0.3221225485        0x8100.2 0x8100.5                                                 default:default
ps0.3221225485                                                        default:default
ps0.3221225486        0x8100.2 0x8100.1                                                 default:default
ps0.3221225486                                                       default:default
ps0.3221225487        0x8100.2 0x8100.2                                                 default:default
ps0.3221225487                                                       default:default

The dhcptester clients are also configured to send BFD echo packets for each client once per second. This generates some (very low) traffic to get some traffic statistics per subscriber interface:

$ monitor interface ps0.3221225487

bng                               Seconds: 8                   Time: 09:18:42
                                                           Delay: 0/0/1
Interface: ps0.3221225487, Enabled, Link is Up
Encapsulation: ENET2
VLAN-Tag [ 0x8100.2 0x8100.2 ]
Remote statistics:
  Input bytes:                      6853 (248 bps)                       [160]
  Output bytes:                     7186 (248 bps)                       [160]
  Input packets:                     206 (0 pps)                           [5]
  Output packets:                    207 (0 pps)                           [5]
Traffic statistics:
  Input bytes:                      6853                                 [160]
  Output bytes:                     7186                                 [160]
  Input packets:                     206                                   [5]
  Output packets:                    207                                   [5]
Protocol: inet, MTU: 1508, Flags: 0x20000000


Without an actual subscriber license, the subscriber scale is very limited, but sufficient for this demo:

mwiget@bng> show system license
License usage:
                                 Licenses     Licenses    Licenses    Expiry
  Feature name                       used    installed      needed
  scale-subscriber                      0           10           0    permanent
  scale-l2tp                            0         1000           0    permanent
  scale-mobile-ip                       0         1000           0    permanent
  VMX-BANDWIDTH                         0       500000           0    60 days
  VMX-SCALE                             3            3           0    59 days
  vmx-subscriber-accounting             0            1           0    60 days
  vmx-subscriber-authentication         0            1           0    60 days
  vmx-subscriber-address-assignment        0         1           0    60 days
  vmx-service-dc                        0            1           0    60 days
  vmx-service-accounting                0            1           0    60 days
  vmx-subscriber-secure-policy          0            1           0    60 days
  vmx-pcrf-subscriber-provisioning        0          1           0    60 days
  vmx-ocs-charging                      0            1           0    60 days
  vmx-nasreq-auth-authorization         0            1           0    60 days
  vmx-service-qos                       0            1           0    60 days
  vmx-service-ancp                      0            1           0    60 days
  vmx-service-cbsp                      0            1           0    60 days

Licenses installed:
  License identifier: E435890758
  License version: 4
  Software Serial Number: 20180209
  Customer ID: vMX-JuniperEval
    vmx-bandwidth-500g - vmx-bandwidth-500g
      count-down, Original validity: 60 days
    vmx-feature-premium - vmx-feature-premium
      count-down, Original validity: 60 days

You can also ping the simulated subscribers from the BNG:

mwiget@bng> ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=32.984 ms
64 bytes from icmp_seq=1 ttl=64 time=36.080 ms
--- ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 32.984/34.532/36.080/1.548 ms


Being able to bring up a simple topology with fully working configs and clients is really powerful. I tend to forget things quickly and often find myself looking back thru my own blog posts ;-). Hopefully this is helpful for others too.

Comments and feedback welcome, as always.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at WordPress.com.

Up ↑

%d bloggers like this: