In WSO2 data center, we run the entire platform on UEC which ships Eucalyptus and we haven’t seen any of the problems we faced with EC2. Ok, private cloud – very important.

As a platform as a service, it’s important to be able to run on top of many IaaS vendors. Same goes for private cloud solutions. So, if Stratos is to be run on OpenStack the auto scaling code should be working. Stratos auto scales on Amazon EC2 that’s the API that’s been implemented. It’s always a pain to support a different API. OpenStack is said to have an EC2 SOAP API but I couldn’t find the extent to which operations are supported. I’ll definitely have a definitive answer if I contact one of the devs. This post however was not about that.

Mark Shuttleworth wrote an excellent blog about some concerns over OpenStack. As Mark correctly observes, it’s not the right thing to innovate at the API level. As eternal perfectionists it’s very easy to get into having cleaner and powerful APIs. Then again from a pragmatic perspective it’s energy that’s better put into where it matters.

WSO2 Stratos already have a working implementation that is tested. Effort of implementing and testing for another API is going to be time consuming. Whether it’s a bad or a good API when we have to get it running we’ll implement it nevertheless.

There are many concerns that needs more attention than having a good API. Higher IO and network throughput comes to the top of the list ‘cos that’s what it matters from a user’s perspective. It’s great that people debating this issue and looking to make a difference!

As a result of Amazon EC2 issues, mainly relating to unacceptable I/O and network delays hosted version of Stratos which we call StratosLive was so damn slow. Now it’s hosted on real hardware and the site is way way faster!

In the long run, enterprise-wide software are better off running on real hardware. Investing in more powerful machines. Or horizontally scale with a number of cheap machines. I can see Amazon’s appeal for a very short term computing problems. Or as Azeez mentioned, cloud bursting for one. Problems like this infamous article about Amazon EC2 trying to solve. Value you gain for paying exuberant bills for Amazon EC2 is very little at the end of the month. Specially when you’re running it long term.

IMO, this is one reason why a reliable private cloud infrastructure matters. In my experience open source private cloud software has a lot to improve. The free stuff by VMware is good for playing around. When you need I/O and network performance + provisioning, you have to pay big bucks.

1. Easiest way to install UEC is to do a CD Install. Follow the instruction on the Ubuntu Wiki to get it installed.
2. Select CLC/Walrus/CC/SC on one machine
3. Install NC in the second machine
4. With the default configuration, the number of instances you can start is very limited. Through the web based admin console, you can configure number of CPUs and memory that goes into small, medium and large instances. Medium instances by default is configured to have 2 CPUs. If your host computer have 8 CPU cores, the number of medium instances you can start is 4. The maximum number of virtual machines KVM support by default is 8 * number of CPU cores. If you’re having 8 cores, KVM can start 64 guests (assuming you have enough memory). So, we need to get this number intu Eucalyptus. On the node controller machine, edit /etc/eucalyptus/eucalypus.conf and add MAX_CORES="64". Or whatever the number applicable to your system.
5. Next, when you install a cloud controller on a computer residing in your LAN, it’ll add another DHCP server. When you start your instances they’ll start getting IPs from the main DHCP server in the LAN. Which is not what you probably want. To prevent this from happening, you have to blacklist MAC addresses generated by UEC. UEC generated MAC addresses begins with d0::0d. Ubuntu wiki contains how to do this for dnsmasq. If you’re using dhcpd you can add,

   class "uec-members" {
       match pick-first-value (option agent.remote-id);
   }
   
   subclass "uec-members" d0:0d:00:00:00:00;
   subclass "uec-members" d0:0d:ff:ff:ff:ff;
   
   subnet ... {
   
        pool {
            deny members of "uec-members";
            range 10.100.1.100 10.100.1.252;
        }
   
    }
   

Voila! You’re on your way to do great things on your private cloud!

Getting it to work in a production setting requires a considerable amount of effort because you’ll be facing undocumented issues. Here, effort refers to scanning through forums and mailing lists.

Still, for me the experience was much pleasant compared to fiddling with some proprietary crap acting as a black box. Although, the documentation for Eucalyptus still a bit rough around the edges; if all else fails you have the option of reading the source to find out the intended behaviour. If you use open source stuff in production I don’t have to say how much time this saves.

After some failed attempts to get this shit up and running, the bazzilion things you should not do when installing UEC turned out to be a useful experience. Doing a CD install, selecting to install cloud stuff from the menu takes you on a short path to finish line. Make sure you select a right topology for your installation.

All the nitty gritty details intentionally left out for a later post.