Nice APIs: Limits in OpenStack SDK

Providing a great experience for Python developers who use Rackspace is my job. In attempting to do that, I've spent time working on, writing, reading, investigating, and dreaming about code that enables our customers to build great things. In the past that has mostly involved Apache's libcloud and Rackspace's own pyrax, as well as several packages offered by various OpenStack services, e.g., python-novaclient.

For the last several months, I've been working with a team on the OpenStack SDK, a project aimed at providing a great experience for Python developers who use OpenStack (hint: that's like the first sentence). Rackspace's platform is built on OpenStack: our users use it, our developers contribute to it, and we want to see it thrive. The application developer story in OpenStack today is not great one, and a group of us on the SDK project are looking to change that. One of the ways we hope to do that is through offering a set of great APIs to work with the many services offered by OpenStack.

While thinking about how Resource classes — our representation of the resources of a REST API — are constructed within the SDK, we've been coming across enough of them that require the ID of another resource that it became a sign to do something about it. Rather than make a user get the ID attribute of one resource and it set it onto another resource, like how a POST /servers requires the IDs of the image and flavor you want, why not just take the resource itself an pull the ID internally before making the HTTP request? Easy enough, right?

While that's cool, it enables something even more cool.

Resource limits, such as how much RAM you've used and are allowed to use, are available through a GET /limits on the compute service. What that returns is a dictionary of absolute limits and of rate limits. The absolute limits are key/value pairs like "totalRamUsed": 1024. Rate limits consist of a list of dictionaries where one of the keys is a list of more dictionaries. It's dictionaries all the way down. Here's a sample.

While reading the limits docs, that change to allow resources to be constructed with other resources popped into my head. What if we make a Limits resource that is constructed from an AbsoluteLimits resource and then a list of RateLimits resources? I'm in.

The code review is available here, but the juicy part is this:

class Limits(resource.Resource):
    base_path = "/limits"
    resource_key = "limits"
    service = compute_service.ComputeService()

    allow_retrieve = True

    absolute = resource.prop("absolute", type=AbsoluteLimits)
    rate = resource.prop("rate", type=list)

Boom. Done. Well, we have to override Resource.get to be able to construct that list of RateLimits, but it was fairly easy. While getting your limits is not in itself some mind blowing task, what the underlying change enabled will make for some very easy to use resources.

Assuming sdk is a Connection instance, getting your RAM usage is as easy as:

>>> limits = sdk.compute.limits()
>>> print(limits.absolute.total_ram_used)
512
>>> print(limits.absolute.total_ram_used / limits.absolute.total_ram)
0.01

That seems pretty basic, and it is, which is ideal. After coding that up, I took a look what other libraries would do to accomplish the same thing.

Unfortunately libcloud only has a limits call in its OpenStack compute v1 API, which I'm unable to use (everything I have access to needs v1.1/v2). However, novaclient certainly supports this.

Assuming nova is a Client instance, getting your RAM usage is as easy as:

limits = nova.limits.get()
ram_limit = filter(lambda l: l.name == "totalRAMUsed", limits.absolute)
print(next(ram_limit).value)

get() returns an object containing rate and absolute attributes, where absolute is a generator of objects with a name and value for each type of limit. I just wanted one of them, so I had to consume a generator and find the one I'm looking for.

What if I wanted to calculate my percentage of usage like before? It actually gets slightly easier. Of course, I could have used this same method for the last example, but this was my process of discovery.

limits = nova.limits.get()
absolute = {l.name: l.value for l in limits.absolute}
print(absolute["totalRAMUsed"] / absolute["maxTotalRAMSize"])

Now I create a dictionary comprehension from the generator of limits. Thanks to that, it's slightly more usable.


We're currently working on how we want the user interfaces to look in OpenStack SDK, so if you have a story to tell in this area, I'd love to hear it. We want to enable people to build great things on top of OpenStack, so email me at brian@python.org and let's see what we can do.

Terry Howe and I have proposed a talk for the OpenStack Summit about building applications with the OpenStack SDK. If that's something you're interested in knowing more about about, check it out.