whatupmiked intersection between networking, not working, and software


Openstack Summit, Barcelona

Openstack Barcelona Summit Videos Openstack Project Navigator

Two years ago I moved over to Europe and started to work with an experienced system admin from the middle-east. Together we talked about how to build-out a lab enviornment for testing customer issues. We went back and forth on design issues and equipment that we would need. Over-all we arrived at a solid environment, for minimum cost, that accounts for the majority of customer use-cases we see. One of our major points of contention was how much to invest in Openstack. As a linux guru my colleague was very excited and eager to work on Openstack issues. As an intermediate linux users I was not as keen.

We ended up investing in some servers for Openstack, but very low-end equipment that can really only handle the minimum use-case for a test-bed… not a cloud. A typical case is a user in need of a source-destination pair for generating various types of TCP/IP traffic. That, or a host for installing a tool being used for a specific customer scenario.

These were Icehouse/Juno days and frankly nobody understood Openstack but the linux guru and the more others would try to understand it the more confused they became. We were devoting about half of our current resources to the Openstack he had built, but the technology represented less than 0.1% of the issues we saw raised by customers. We left the Openstack in place. We didn’t have the environment any-where else and the deployment over-head for our linux guru was high-enough that he did not want to spend time rebuilding the environment the next time we might see an Openstack issue.

Since that initial build we have been leary of devoting additional resources to Openstack because: a) The complexity is daunting. b) The resource requirements to account for all the snow-flakes is high. c) The user-uptake we see is low.

I consider myself an operator. Working to support customers I view issues from the perspective of a troubleshooter. In that role I get to see many of the customer pain-points. Two years ago I did not understand Openstack at all, I didn’t understand the use-case, and I thought it was far to complex and unneccesary for a network engineer to deal with.

Since that time I’ve been working alot more with SDN equipment, attended various training, some that I would highly recommend, stood up an Openstack Mitaka, and attended Openstack Summit Barcelona. After this I understand the Openstack use-case better. I find it slightly less complex and still think it is largely irrelevant to a network engineer. For a devops engineer or sysadmin, I feel Openstack is highly relevant.

In the Openstack Barcelona Summit I saw: a) Multiple presentations reflecting the same pain and suffering that we experienced during deployment. b) Several very good summaries that drove home the use-case for me. c) Alot of one-off projects that make no sense to me as a network operator. d) A few troubleshooting case-studies that I found very valuable. e) A couple of presentations which explained to me why people are even bothering with it.

Linux has been on the rise for 15 years now. Many operators, developers and users have grown up using linux and been highly rewarded for their efforts. They’ve seen hyper-scales take linux servers and put them in the back-bone of all major web-services. They have seen what the hyper-scales can do and want to replicate that success. They have outgrown the tool-set of linux on a single-server and want to apply all of the same skills and principles to hundreds of servers. The analogy pets vs. cattle was used in 3 presentations I attended.

What is Openstack?

Openstack is a distributed linux OS. All the services you would pack into a single computer are now distributed across multiple computers that commincate with each other through REST (http requests) calls. The services are all written using python.

Openstack = Cloud Linux
REST calls = IPC
Services = Python scripts
Underlying OS = Linux

Complexity and Deployment

Since we first started using Openstack a miriad of deployment methods have popped up. I count at least eight (vanilla, RDOProject, devstack, packstack, ansible, puppet, chef) and three of which are managed deployments from major vendors (Redhat, Suse, Canonical).

While it is great that all of these deployment methods still exist, if I had used a scripted install when I initially deployed it I would have been lost during many of the talks and un-equipped for any meaningful customer interaction. I firmly believe, and this was echoed by others during the conference, that anyone planning to use or troubleshoot Openstack should go through at least one vanilla deployment to gain some understanding of the various compenents, where they are configured, and how they interact.

There was no vendor or individual I heard at the conference who thought that Openstack was simple. In fact a theme of many talks was the complexity and pain points involved. I heard of multiple side projects and tools designed to reduce the complexity… of course each tool has it’s own level of complexity as well. Openstack engineers are expensive it seems and that’s not an accident, the skill-set required to understand all the components, bring up a cloud, and actually operate it is broad.

You have to go through a Vanilla reference install to get a handle on all the components.

Why bother in the first place?

There was a great presentation from internap and catalyst cloud, they put together some numbers around the cost to build, own an operate a private cloud. It seems that AWS has a high margin… and if you go the full opensource route (like cataylst) you can get a 5x price reduction. That is, if you have the skill-set to go full opensource.

They provided a spread-sheet to tweak for calculating your total cost of ownership (TCO) and I plan to play around with it more. The long and short of it though is if you can build it, you can save quite a bit versus public cloud options.

Favorite talks from the summit

If it’s blurry change your video settings to 480p under the youtube video settings.

Things you MUST know before you deploy Openstack -Bruno Lago Can openstack beat AWS in price? Vanilla or Distributions How do they differentiate? Openstack Troubleshooting, So simple even your kids can do it Troubleshooting Neutron: Physical and Virtual Networks Anatomy of Openstack Neutron Through the Eagle Eyes of Troubleshooters DevOops: Lessons Learned from a Cloud Network Architect - Rackspace talking about networking errors they hit and how they dealt with them. Brokenstack: Openstack Failure Stories - Cisco talks through some deployments fails they had with really big customers (Comcast) Ceph, Now and Later - Our Plan for Open Unified Cloud Storage - Lead for the storage driver to Openstack talks through it. Got over-my-head very quickly but may be insightful for storage expert. Brocade: StackStorm and Openstack: Two Practical Cases for Event Driven Automation