Kolla: Openstack in Docker containers with Ansible playbooks

Kolla: Openstack in Docker containers with Ansible playbooks

Deploying Openstack isn’t something you do before your first coffee in the morning. There are many choices, from what version and distribution to which Linux distribution and version to run it on. Once that’s decided, the painful task of checking the major and minor release dependencies starts. More often than not, this led to a full re-install of the host OS, making sure no automatic package updating gets triggered.
Having played a lot with Docker, I was sure someone else must have added 1+1 together and looked into taming Openstack into Containers. Well, turns out I was right. I found the following presentation with video from the Openstack Summit in Tokyo 2015:

Kolla: Ansible Deployment + OpenStack in Docker Containers = Operator Bliss

The main Kolla website can be found here: https://wiki.openstack.org/wiki/Kolla

I had to experience this for myself, so I picked a running server that I have been using for some Docker related work that runs Ubuntu 16.04.1 LTS. The following notes describe the individual steps in getting Openstack Kilo up and running. The quick start guide from Kolla http://docs.openstack.org/developer/kolla/quickstart.html has been very helpful in bringing up an “all-in-one” setup and I document here only area’s I initially struggled with:

1. Host machine with 2 (or more) interfaces

The recommended deployment target requirements besides memory and disk space are 2 (or more) network interfaces. My server only has a single public interface, so I followed the hint about using a virtual ethernet veth pair and bridging one veth with the single public eth0 interface. To make this change permanent, I wanted this to happen via /etc/network/interfaces, which requires bridge-utils package to be installed:

apt-get bridge-utils

Content of the updated /etc/network/interfaces:

### Hetzner Online GmbH - installimage

# Loopback device:
auto lo
iface lo inet loopback
iface lo inet6 loopback

auto  eth0
auto br0
iface br0 inet static
  pre-up ip link add veth0 type veth peer name veth1
  pre-up ifconfig veth0 up
  pre-up ifconfig veth1 up
  bridge_ports eth0 veth0
  bridge_fd 0
  bridge_maxwait 0
  address   138.201.131.167
  netmask   255.255.255.192
  gateway   138.201.131.129
  # default route to access subnet
  up route add -net 138.201.131.128 netmask 255.255.255.192 gw 138.201.131.129 br0

iface br0 inet6 static
  address 2a01:4f8:172:2ee6::2
  netmask 64
  gateway fe80::1

2. /etc/kolla/globals.yml

After following all the steps from the quick start guide, it was time to make the required changes to globals.yml. I wanted to take advantage of prebuilt packages, so opted for the tag 2.0.2. The default ‘3.0.0’ failed, forcing me to look a bit more into the source of these tags. I found the tags on https://github.com/openstack/kolla and 2.0.2 seemed like a good choice. The quick start guide recommends to run ‘kolla-ansible prechecks’, which I used extensively. I got stuck twice, once when it failed with finding a free port for Horizon and once about being able to ping an address (yes, it failed because the ping succeeded). The first problem with Horizon was that I still had a web server running on port 80, though I only figured this out by looking into the task file https://github.com/openstack/kolla/blob/master/ansible/roles/prechecks/tasks/port_checks.yml. The second issue was a bit harder to resolve, because I couldn’t simply pick a valid free IPv4 address due to my rented server only having one address. Some research revealed an unsupported setting ’enable_haproxy: “no”, that takes care of it.

The winning combo for my server is shown as unified diff:

$ diff -u etc/kolla/globals.yml /etc/kolla/globals.yml 
--- etc/kolla/globals.yml 2016-08-13 18:33:47.089230761 +0200
+++ /etc/kolla/globals.yml 2016-08-14 10:43:24.668329507 +0200
@@ -17,13 +17,16 @@
#kolla_install_type: "binary"

# Valid option is Docker repository tag
-#openstack_release: "3.0.0"
+openstack_release: "2.0.2"

# This should be a VIP, an unused IP on your network that will float between
# the hosts running keepalived for high-availability. When running an All-In-One
# without haproxy and keepalived, this should be the first IP on your
# 'network_interface' as set in the Networking section below.
-kolla_internal_vip_address: "10.10.10.254"
+kolla_internal_vip_address: "138.201.131.167"
+
+# don't have spare IP on the server in the public cloud
+enable_haproxy: "no"

# This is the DNS name that maps to the kolla_internal_vip_address VIP. By
# default it is the same as kolla_internal_vip_address.
@@ -58,7 +61,7 @@
# This interface is what all your api services will be bound to by default.
# Additionally, all vxlan/tunnel and storage network traffic will go over this
# interface by default. This interface must contain an IPv4 address.
-network_interface: "eth0"
+network_interface: "br0"

# These can be adjusted for even more customization. The default is the same as
# the 'network_interface'. These interfaces must contain an IPv4 address.
@@ -72,10 +75,10 @@
# though an IP address can exist on this interface, it will be unusable in most
# configurations. It is recommended this interface not be configured with any IP
# addresses for that reason.
-neutron_external_interface: "eth1"
+neutron_external_interface: "veth1"

# Valid options are [ openvswitch, linuxbridge ]
-#neutron_plugin_agent: "openvswitch"
+neutron_plugin_agent: "linuxbridge"

With above changes, ‘kolla-ansible prechecks’ ran without complaints and I was able to successfully deploy Openstack with ‘kolla-ansible deploy’.

The quick start guide also gives good guidance about cleaning up a failed deployment:

tools/cleanup-containers
tools/cleanup-host

3. Final steps

Only now I realized, that I never set any password, particularly none for Horizon. Well, turns out, there was a step in the quick start guide, ‘kolla-genpwd’, that generated all of them. There is a post deploy ansible script that generates the openrc file and another script to populate glance with an image and a neutron network:

kolla-ansible post-deploy
. /etc/kolla/admin-openrc.sh
kolla/tools/init-runonce

4. Trying it out

Logged into Horizon as user admin in Domain default, using the password from the admin-openrc.sh file. Launched successfully two cirros images on a test virtual network.

Heureka! All worked. Now lets look at the running containers, 27 in total:

They are using the following, pre-built Docker images pulled from the Docker Hub:

mwiget@sa:~$ docker images |grep centos
kolla/centos-binary-heat-api                    2.0.2               4b51565bfeec        2 weeks ago         571.2 MB
kolla/centos-binary-heat-engine                 2.0.2               e685e9910b38        2 weeks ago         571.2 MB
kolla/centos-binary-heat-api-cfn                2.0.2               8fbd090ffd21        2 weeks ago         571.2 MB
kolla/centos-binary-glance-api                  2.0.2               5cf83b7036d5        2 weeks ago         667.3 MB
kolla/centos-binary-glance-registry             2.0.2               cd02965fd258        2 weeks ago         667.3 MB
kolla/centos-binary-nova-libvirt                2.0.2               f9424d742fa9        2 weeks ago         1.106 GB
kolla/centos-binary-neutron-server              2.0.2               a7036b58ef1a        2 weeks ago         797.1 MB
kolla/centos-binary-neutron-metadata-agent      2.0.2               2b1ccb3cd1ad        2 weeks ago         797.1 MB
kolla/centos-binary-nova-compute                2.0.2               5377b0621ee7        2 weeks ago         1.055 GB
kolla/centos-binary-neutron-openvswitch-agent   2.0.2               cdbf1f6bd5ce        2 weeks ago         822.7 MB
kolla/centos-binary-neutron-linuxbridge-agent   2.0.2               21881c1d2370        2 weeks ago         822.4 MB
kolla/centos-binary-neutron-dhcp-agent          2.0.2               2e36714d61ff        2 weeks ago         797.1 MB
kolla/centos-binary-neutron-l3-agent            2.0.2               2e36714d61ff        2 weeks ago         797.1 MB
kolla/centos-binary-nova-conductor              2.0.2               eed1e857e3ba        2 weeks ago         609.3 MB
kolla/centos-binary-nova-consoleauth            2.0.2               5a3ba91867a6        2 weeks ago         609.4 MB
kolla/centos-binary-nova-api                    2.0.2               1c5d9466bc83        2 weeks ago         609.3 MB
kolla/centos-binary-nova-scheduler              2.0.2               318f7d4863fe        2 weeks ago         609.3 MB
kolla/centos-binary-nova-ssh                    2.0.2               484d955bf16b        2 weeks ago         610.2 MB
kolla/centos-binary-nova-novncproxy             2.0.2               f52035c3f025        2 weeks ago         609.9 MB
kolla/centos-binary-horizon                     2.0.2               8d66a658a7eb        2 weeks ago         703 MB
kolla/centos-binary-keystone                    2.0.2               d55fac3e03d2        2 weeks ago         585.1 MB
kolla/centos-binary-kolla-toolbox               2.0.2               b353561584ef        2 weeks ago         636.4 MB
kolla/centos-binary-openvswitch-vswitchd        2.0.2               c20d4066ccf7        2 weeks ago         330.5 MB
kolla/centos-binary-openvswitch-db-server       2.0.2               1eda2c04df04        2 weeks ago         330.5 MB
kolla/centos-binary-mariadb                     2.0.2               369d5bf095f3        2 weeks ago         630 MB
kolla/centos-binary-rabbitmq                    2.0.2               ce33162566c9        2 weeks ago         370.2 MB
kolla/centos-binary-memcached                   2.0.2               2fc1d618ac62        2 weeks ago         354.5 MB
kolla/centos-binary-heka                        2.0.2               b6a0ff4a1ca9        2 weeks ago         371 MB
kolla/centos-binary-cron                        2.0.2               7237ad13540f        2 weeks ago         317.4 MB

5. Summary

Well, this has been a rather pleasant experience, so kudos to the maintainers and authors of Kolla !! I’m sure I’ll be using this method of deploying Openstack more in the future. If you read that far, you realise that I’m running Openstack Containers using centos binaries on a server running Ubuntu 16.04. How cool is that!

Being able to install Openstack, take it for a spin and clean it up again is real win and timesaver.