Extending the OpenStack - Part I: APIs

If OpenStack is something you have heard of, but aren’t too familiar with, you probably think “virtual servers” - or maybe “automatic server provisioning”. That is in my opinion not what OpenStack is about. It is about being able to integrate with “everything infrastructure-related”, and enforcing access in a secure manner. OpenStack is a very good starting point for achieving that, giving you virtual servers, networking, storage management, and at it’s core: identity management. Installing virtual servers and provisioning resources for them are well and good, but that’s just an appliance. The fun part comes when we can Build Things.

One of the OpenStack-projects named Keystone, provide identity management. This component is responsible for dealing with the service catalogue and API endpoints in your OpenStack cloud. User authentication happen through this, and users usually have roles assigned on one or more projects. When an end user want to create a virtual server, the user passes an authentication token to the compute service API (nova), which it verifies with keystone before performing any actions. Through this mechanism, user authentication and authorization is nicely decoupled from the services that end up delivering infrastructure. It is meant to be used as a mean to extend OpenStack, and it’s pretty much straight forward.

Prerequisites

  • An OpenStack-solution
  • Ability to write a small web service application, for example using something like Flask. Can be done in any programming language, but python give you access to a bunch of SDKs for speaking with the OpenStack APIs which can come in handy.
  • Something you want to implement

So if you have those three things, you’re good to go. If you don’t have those - but wanna play around and see what you can do, I can push you in the right direction to get started.

Preparing OpenStack

The two first steps of adding the API to the service and endpoint catalogue aren’t mandatory - but it’s a polite thing to do. By doing so, users of the cloud can see the URLs where your API is available.

## As an admin user, perform the following steps:

# openstack service create myservice --name myservice
+---------+----------------------------------+
| Field   | Value                            |
+---------+----------------------------------+
| enabled | True                             |
| id      | b81a620dfdb54592b97c67816d00fc43 |
| name    | myservice                        |
| type    | myservice                        |
+---------+----------------------------------+

# openstack endpoint create myservice admin http://ctrl-node.mycloud.example.org:1337
# openstack endpoint create myservice public http://ctrl-node.mycloud.example.org:1337
# openstack endpoint create myservice internal http://ctrl-node.mycloud.example.org:1337
[...]

# openstack endpoint list --service myservice
+----------------------------------+--------+--------------+--------------+---------+-----------+-----------------------------------------------+
| ID                               | Region | Service Name | Service Type | Enabled | Interface | URL                                           |
+----------------------------------+--------+--------------+--------------+---------+-----------+-----------------------------------------------+
| 1fbbbcc57108422596e07ab4bd713631 | None   | myservice    | myservice    | True    | internal  | http://ctrl-node.mycloud.example.org:1337     |
| 25edd1c28178462980e90bce2b8140f2 | None   | myservice    | myservice    | True    | public    | http://ctrl-node.mycloud.example.org:1337     |
| 78b2090f1df4450fb4544abf1eee842a | None   | myservice    | myservice    | True    | admin     | http://ctrl-node.mycloud.example.org:1337     |
+----------------------------------+--------+--------------+--------------+---------+-----------+-----------------------------------------------+

The more important step is to create a service user, which will be used to perform administrative tasks - such as validating authentication tokens.

# openstack user create --email myservice@localhost --password myservicesecretpassword myservice
+---------------------+----------------------------------+
| Field               | Value                            |
+---------------------+----------------------------------+
| domain_id           | default                          |
| email               | myservice@localhost              |
| enabled             | True                             |
| id                  | ab3e5a51a87a44898c8543e0a254bd3b |
| name                | myservice                        |
| options             | {}                               |
| password_expires_at | None                             |
+---------------------+----------------------------------+

# openstack role add --project services --user myservice admin

At this point, we have our service user.

The web service

You need to handle the API calls, and you need to verify authenticated requests.

Validating a token can be done with the following code:

from keystoneauth1.identity import v3
from keystoneauth1 import session
from keystoneclient.v3 import client

# Authenticate with the service user, and create a keystone client
auth = v3.Password(auth_url='https://ctrl-node.mycloud.example.org:35357/v3',
                   username='myservice',
                   password='myservicesecretpassword',
                   project_name='services',
                   user_domain_id='default',
                   project_domain_id='default')
sess = session.Session(auth=auth)
keystone = client.Client(session=sess)

# Validator-class that can easily be extended with functions
# that test role/project-access. self.validate contains a hash
# with information about this.
class Validator:
    def __init__(self, token):
        self.validate = keystone.tokens.validate(token)

Tokens are passed between OpenStack-services using the x-auth-token http-header. In Flask, picking this out in your service routes can be as simple as request.headers.get('x-auth-token').

Using the validator-class above in a route can be

from flask import Flask
from flask import request
from flask import abort
from flask import jsonify

from validator import Validator
from thing import Thing

# A helper function to wrap the raised exception in the case 
# of an invalid token.
def create_validator():
    try:
        v = Validator(request.headers.get('x-auth-token'))
    except:
        abort(403)
    return v

# Routes below.
# The /-route can be used as a liveliness and ready probe.
@app.route("/")
def check():
    return jsonify({"myservice": 0.1})

# Authenticated route.  Passing the validator object to the
# constructor of Thing(), and calling a method - returning its
# output as json.
@app.route("/v1/thing/list")
def thing_list():
    return jsonify(Thing(create_validator()).list_things())

What is Thing()? It’s a class we’ve made to split application logic from routes. At the route level, we have ensured that the request is authenticated now we want to do something.

class Thing:
    def __init__(self, validator):
        self.v = validator

    def list_things(self):
        return { 'things': ['apple', 'chair', 'banana', 'television'] }

The Thing class takes a validator in its constructor for two reasons.

  1. It’s nice to authenticate the request early on.
  2. You get access to role and project-information from within the class, and can make decisions based on that.

A slightly more complete version of this skeleton is available on GitHub.

Testing it out

Loading our liveliness route works fine without any authentication.

$ curl http://ctrl-node.mycloud.example.org:1337/
{
  "myservice": 0.1
}

Our web service for listing things requires authentication:

$ curl http://ctrl-node.mycloud.example.org:1337/v1/thing/list
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>403 Forbidden</title>
<h1>Forbidden</h1>
<p>You don't have the permission to access the requested resource. It is either read-protected or not readable by the server.</p>

So we need to issue a token. We can do that by calling the identity API manually, or if one have set up a CLI to our cloud - one can use that to issue a token.

$ openstack token issue
+------------+----------------------------------+
| Field      | Value                            |
+------------+----------------------------------+
| expires    | 2018-01-27T23:23:16+0000         |
| id         | 39465e6698b7409cb926a870cef1e5bb |
| project_id | dafe2c2a6d6e4bfc9146ea975b2897e0 |
| user_id    | 830ec37bc8774738904731a7bd55cb58 |
+------------+----------------------------------+

Inserting the token id as the http header ‘x-auth-token’ will authenticate requests against our OpenStack cloud, including our custom built web service:

$ curl -H 'x-auth-token: 39465e6698b7409cb926a870cef1e5bb' http://ctrl-node.mycloud.example.org:1337/v1/thing/list
{
  "things": [
    "apple",
    "chair",
    "banana",
    "television"
  ]
}

The web service has the information it needs about the calling user by validating the token. Which user, domain, project, and what roles the user have.

At this point one are free to implement whatever one pleases. One can integrate with some in-house system that shows invoice information for a customer - or may just be to create some helper APIs that simply re-uses the provided token to chain-run a series of other OpenStack APIs.

If you want to provide a user friendly web-interface for your APIs, stay tuned for Part II.

Links