Kolla: Openstack in Docker containers with Ansible playbooks

Kolla: Openstack in Docker containers with Ansible playbooks

Deploying Openstack isn’t something you do before your first coffee in the morning. There are many choices, from what version and distribution to which Linux distribution and version to run it on. Once that’s decided, the painful task of checking the major and minor release dependencies starts. More often than not, this led to a full re-install of the host OS, making sure no automatic package updating gets triggered.
Having played a lot with Docker, I was sure someone else must have added 1+1 together and looked into taming Openstack into Containers. Well, turns out I was right. I found the following presentation with video from the Openstack Summit in Tokyo 2015:

Kolla: Ansible Deployment + OpenStack in Docker Containers = Operator Bliss

The main Kolla website can be found here: https://wiki.openstack.org/wiki/Kolla

I had to experience this for myself, so I picked a running server that I have been using for some Docker related work that runs Ubuntu 16.04.1 LTS. The following notes describe the individual steps in getting Openstack Kilo up and running. The quick start guide from Kolla http://docs.openstack.org/developer/kolla/quickstart.html has been very helpful in bringing up an “all-in-one” setup and I document here only area’s I initially struggled with:

1. Host machine with 2 (or more) interfaces

The recommended deployment target requirements besides memory and disk space are 2 (or more) network interfaces. My server only has a single public interface, so I followed the hint about using a virtual ethernet veth pair and bridging one veth with the single public eth0 interface. To make this change permanent, I wanted this to happen via /etc/network/interfaces, which requires bridge-utils package to be installed:

apt-get bridge-utils

Content of the updated /etc/network/interfaces:

### Hetzner Online GmbH - installimage

# Loopback device:
auto lo
iface lo inet loopback
iface lo inet6 loopback

auto  eth0
auto br0
iface br0 inet static
  pre-up ip link add veth0 type veth peer name veth1
  pre-up ifconfig veth0 up
  pre-up ifconfig veth1 up
  bridge_ports eth0 veth0
  bridge_fd 0
  bridge_maxwait 0
  # default route to access subnet
  up route add -net netmask gw br0

iface br0 inet6 static
  address 2a01:4f8:172:2ee6::2
  netmask 64
  gateway fe80::1

2. /etc/kolla/globals.yml

After following all the steps from the quick start guide, it was time to make the required changes to globals.yml. I wanted to take advantage of prebuilt packages, so opted for the tag 2.0.2. The default ‘3.0.0’ failed, forcing me to look a bit more into the source of these tags. I found the tags on https://github.com/openstack/kolla and 2.0.2 seemed like a good choice. The quick start guide recommends to run ‘kolla-ansible prechecks’, which I used extensively. I got stuck twice, once when it failed with finding a free port for Horizon and once about being able to ping an address (yes, it failed because the ping succeeded). The first problem with Horizon was that I still had a web server running on port 80, though I only figured this out by looking into the task file https://github.com/openstack/kolla/blob/master/ansible/roles/prechecks/tasks/port_checks.yml. The second issue was a bit harder to resolve, because I couldn’t simply pick a valid free IPv4 address due to my rented server only having one address. Some research revealed an unsupported setting ’enable_haproxy: “no”, that takes care of it.

The winning combo for my server is shown as unified diff:

$ diff -u etc/kolla/globals.yml /etc/kolla/globals.yml 
--- etc/kolla/globals.yml 2016-08-13 18:33:47.089230761 +0200
+++ /etc/kolla/globals.yml 2016-08-14 10:43:24.668329507 +0200
@@ -17,13 +17,16 @@
#kolla_install_type: "binary"

# Valid option is Docker repository tag
-#openstack_release: "3.0.0"
+openstack_release: "2.0.2"

# This should be a VIP, an unused IP on your network that will float between
# the hosts running keepalived for high-availability. When running an All-In-One
# without haproxy and keepalived, this should be the first IP on your
# 'network_interface' as set in the Networking section below.
-kolla_internal_vip_address: ""
+kolla_internal_vip_address: ""
+# don't have spare IP on the server in the public cloud
+enable_haproxy: "no"

# This is the DNS name that maps to the kolla_internal_vip_address VIP. By
# default it is the same as kolla_internal_vip_address.
@@ -58,7 +61,7 @@
# This interface is what all your api services will be bound to by default.
# Additionally, all vxlan/tunnel and storage network traffic will go over this
# interface by default. This interface must contain an IPv4 address.
-network_interface: "eth0"
+network_interface: "br0"

# These can be adjusted for even more customization. The default is the same as
# the 'network_interface'. These interfaces must contain an IPv4 address.
@@ -72,10 +75,10 @@
# though an IP address can exist on this interface, it will be unusable in most
# configurations. It is recommended this interface not be configured with any IP
# addresses for that reason.
-neutron_external_interface: "eth1"
+neutron_external_interface: "veth1"

# Valid options are [ openvswitch, linuxbridge ]
-#neutron_plugin_agent: "openvswitch"
+neutron_plugin_agent: "linuxbridge"

With above changes, ‘kolla-ansible prechecks’ ran without complaints and I was able to successfully deploy Openstack with ‘kolla-ansible deploy’.

The quick start guide also gives good guidance about cleaning up a failed deployment:


3. Final steps

Only now I realized, that I never set any password, particularly none for Horizon. Well, turns out, there was a step in the quick start guide, ‘kolla-genpwd’, that generated all of them. There is a post deploy ansible script that generates the openrc file and another script to populate glance with an image and a neutron network:

kolla-ansible post-deploy
. /etc/kolla/admin-openrc.sh

4. Trying it out

Logged into Horizon as user admin in Domain default, using the password from the admin-openrc.sh file. Launched successfully two cirros images on a test virtual network.

Openstack Horizon UI
Openstack Horizon UI

Heureka! All worked. Now lets look at the running containers, 27 in total:

Running Containers
Running Containers

They are using the following, pre-built Docker images pulled from the Docker Hub:

mwiget@sa:~$ docker images |grep centos
kolla/centos-binary-heat-api                    2.0.2               4b51565bfeec        2 weeks ago         571.2 MB
kolla/centos-binary-heat-engine                 2.0.2               e685e9910b38        2 weeks ago         571.2 MB
kolla/centos-binary-heat-api-cfn                2.0.2               8fbd090ffd21        2 weeks ago         571.2 MB
kolla/centos-binary-glance-api                  2.0.2               5cf83b7036d5        2 weeks ago         667.3 MB
kolla/centos-binary-glance-registry             2.0.2               cd02965fd258        2 weeks ago         667.3 MB
kolla/centos-binary-nova-libvirt                2.0.2               f9424d742fa9        2 weeks ago         1.106 GB
kolla/centos-binary-neutron-server              2.0.2               a7036b58ef1a        2 weeks ago         797.1 MB
kolla/centos-binary-neutron-metadata-agent      2.0.2               2b1ccb3cd1ad        2 weeks ago         797.1 MB
kolla/centos-binary-nova-compute                2.0.2               5377b0621ee7        2 weeks ago         1.055 GB
kolla/centos-binary-neutron-openvswitch-agent   2.0.2               cdbf1f6bd5ce        2 weeks ago         822.7 MB
kolla/centos-binary-neutron-linuxbridge-agent   2.0.2               21881c1d2370        2 weeks ago         822.4 MB
kolla/centos-binary-neutron-dhcp-agent          2.0.2               2e36714d61ff        2 weeks ago         797.1 MB
kolla/centos-binary-neutron-l3-agent            2.0.2               2e36714d61ff        2 weeks ago         797.1 MB
kolla/centos-binary-nova-conductor              2.0.2               eed1e857e3ba        2 weeks ago         609.3 MB
kolla/centos-binary-nova-consoleauth            2.0.2               5a3ba91867a6        2 weeks ago         609.4 MB
kolla/centos-binary-nova-api                    2.0.2               1c5d9466bc83        2 weeks ago         609.3 MB
kolla/centos-binary-nova-scheduler              2.0.2               318f7d4863fe        2 weeks ago         609.3 MB
kolla/centos-binary-nova-ssh                    2.0.2               484d955bf16b        2 weeks ago         610.2 MB
kolla/centos-binary-nova-novncproxy             2.0.2               f52035c3f025        2 weeks ago         609.9 MB
kolla/centos-binary-horizon                     2.0.2               8d66a658a7eb        2 weeks ago         703 MB
kolla/centos-binary-keystone                    2.0.2               d55fac3e03d2        2 weeks ago         585.1 MB
kolla/centos-binary-kolla-toolbox               2.0.2               b353561584ef        2 weeks ago         636.4 MB
kolla/centos-binary-openvswitch-vswitchd        2.0.2               c20d4066ccf7        2 weeks ago         330.5 MB
kolla/centos-binary-openvswitch-db-server       2.0.2               1eda2c04df04        2 weeks ago         330.5 MB
kolla/centos-binary-mariadb                     2.0.2               369d5bf095f3        2 weeks ago         630 MB
kolla/centos-binary-rabbitmq                    2.0.2               ce33162566c9        2 weeks ago         370.2 MB
kolla/centos-binary-memcached                   2.0.2               2fc1d618ac62        2 weeks ago         354.5 MB
kolla/centos-binary-heka                        2.0.2               b6a0ff4a1ca9        2 weeks ago         371 MB
kolla/centos-binary-cron                        2.0.2               7237ad13540f        2 weeks ago         317.4 MB

5. Summary

Well, this has been a rather pleasant experience, so kudos to the maintainers and authors of Kolla !! I’m sure I’ll be using this method of deploying Openstack more in the future. If you read that far, you realise that I’m running Openstack Containers using centos binaries on a server running Ubuntu 16.04. How cool is that!

Being able to install Openstack, take it for a spin and clean it up again is real win and timesaver.

12 thoughts on “Kolla: Openstack in Docker containers with Ansible playbooks

Add yours

  1. Hi, I’d like to try kolla on my macmini, below is my network settings

    Could you suggest me how to make the bridge for veth pair?
    Cause I don’t understand above network setting you mentioned 😦
    in your `/etc/network/interfaces` file, the eth0 is the original interface with the static ip( for you to access external internet?
    and now you use br0 to access external internet, right?


    1. Correct. That IP was assigned previously to eth0. The interfaces content creates bridge br0, creates the veth pair, and places the eth0 together with one veth interfae into the bridge br0. Then the public static IP is given to the bridge instead of eth0.


    1. Looks like you use ppp0 to connect to the Internet, not via enp1s0f0, so no need to modify the default route. In my case the routing was done via eth0, hence I needed to adjust the default route.


  2. Hi mwiget, this is almost the only “hint” of how to deploy OpenStack with Kolla on Hetzner server. Many thanks! I have to say that Hetzner + OpenStack is the most horrible combination I have ever met, especially the networking configuration part. I tried with your recommended settings, and the installation process has no problem. But the issue is neither the qrouter nor the created VM instances can access the internet. I know it’s just your experiment a couple of years ago. But could you remember if your created cirros VMs can access internet by any chance? Thanks a lot!


    1. Thanks. I don’t think I had Internet access from a VM, that wasn’t the point back then, I used it more like an internal lab. But I moved on to Containers since then 😉


      1. Thank you, mwiget! As a toy box, that’s totally enough. What if I want the VMs be able to “see” the outside world? What extra configs are necessary for that? Could you shed some lights?


  3. Thanks for the article. Good information here.

    Just an FYI, all links contained in the post (except for the first one) point back to this post instead of to their actual targets (GitHub links point to this page, etc).


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at WordPress.com.

Up ↑

%d bloggers like this: