My setup was,

• Cassandra running in a VirtualBox VM with default parameters. Only the data directories were changed
• VM was running Kubuntu 11.04
• JDK 1.6.0_27
• JVM was not warmed up before carrying out the test. I started Cassandra with an empty keyspace for each case
• I was testing the code hosted here

Here’s how the three column families look like.

REGData column family,

PropertyIndex column family,

TagIndex column family,

Here are the results,

The graphs looks very similar. However Cassandra 1.0.2 is has faster write speeds. Here are the raw data. T in the graph represent number of tags and P number of properties.

Sudden spike in write speed increase is a bit scary. That can be due to I/O bottleneck in the machine. At that point my disk started grinding heavily. During the time of the tests, I didn’t do any other disk intensive tasks. On server grade hardware with a few fast disks results might be different. Radically even.

1. Certificates. We’ll be using two certificates for signing. We’ll create a private/public keypair in the client side and import the client’s public key to server. Likewise, we’ll import server’s public key to the machine running the client.
2. First let’s create a key pair to be used in the client,

   makecert -r -pe -n "CN=mycert" -sky exchange -sv mycert.pvk mycert.cer
   

   This will create a private and public key pair. Next we need to import this into Windows certificate store. In order to do that we need to convert our key pair to PFX format which includes both the private and the public key,

   pvk2pfx -pvk mycert.pvk -spc mycert.cer -pfx mycert.pfx
   

   Now, run mmc and add a certificate snap-in for the Local Computer. Browse into Trusted People and import mycert.pfx. Just click next when it asks for the private key password.

3. We have set the certificates for the client. Next we have to extract server’s public key and import it into Windows certificate store. Since I’m going to use WSO2 ESB for the server side, we need to extract the public key from the Java keystore that’s being used by the ESB. Browse to <ESB HOME>\repository\resources\security and give the following command. Java needs to be in your PATH. When prompted type “wso2carbon” as the keystore password.

   keytool -keystore wso2carbon.jks -export -alias localhost -file localhost.cer
   

   Import localhost.cer into Trusted People just like before.

4. Now we need to import the client’s public key to the server. Start the ESB by double clicking wso2server.bat. Browse into https://localhost:9443 and login with admin/admin. Click Configure -> Key Stores.

   

   Click Import Cert

   

   and browse and select mycert.cer we just created and click Import.

   

5. We’re done setting up certificates. Let’s create a simple secure service. Luckily ESB ships an echo service which when you send a message, echoes it back. Click Main -> List (under Web Services).

   

   Here you see a list of web services. In front of the echo service you’ll see a link saying “Unsecured”.

   

   Click it and select “yes” from the drop down to apply security for the echo service.

   

   Here you’ll see a list of pre-configured security scenarios. We’ll be using number 2, Non-Repudiation under Basic Scenarios. Select it and click Next.

   

   Select wso2carbon.jks as a trusted keystore and click Finish. Echo service is secured now.

   

6. We created and setup certificates and now have a secured service. Open up Visual Studio and create a new console application. You can create any kind of project but I’d prefer to create console apps for testing these type of scenarios.
7. Add a Service Reference to your project. WSDL file for the service is located at http://localhost:8280/services/echo?wsdl
8. For the binding that you’ll be using you have to configure security,

   <security defaultAlgorithmSuite="Default" authenticationMode="MutualCertificateDuplex"
       requireDerivedKeys="false" securityHeaderLayout="Lax" includeTimestamp="true"
       keyEntropyMode="CombinedEntropy" messageProtectionOrder="SignBeforeEncrypt"
       messageSecurityVersion="Default" requireSignatureConfirmation="false">
   <localClientSettings cacheCookies="true" detectReplays="true"
       replayCacheSize="900000" maxClockSkew="00:05:00" maxCookieCachingTime="Infinite"
       replayWindow="00:05:00" sessionKeyRenewalInterval="10:00:00"
       sessionKeyRolloverInterval="00:05:00" reconnectTransportOnFailure="true"
       timestampValidityDuration="00:05:00" cookieRenewalThresholdPercentage="60" />
   <localServiceSettings detectReplays="true" issuedCookieLifetime="10:00:00"
       maxStatefulNegotiations="128" replayCacheSize="900000" maxClockSkew="00:05:00"
       negotiationTimeout="00:01:00" replayWindow="00:05:00" inactivityTimeout="00:02:00"
       sessionKeyRenewalInterval="15:00:00" sessionKeyRolloverInterval="00:05:00"
       reconnectTransportOnFailure="true" maxPendingSessions="128"
       maxCachedCookies="1000" timestampValidityDuration="00:05:00" />
   <secureConversationBootstrap />
   </security>
   

9. Your actual service should looks like this,

   static void Main(string[] args)
   {
       svc.echoPortTypeClient echo = new svc.echoPortTypeClient("echoHttpSoap11Endpoint");
   
       echo.ClientCredentials.ServiceCertificate.SetDefaultCertificate(
          System.Security.Cryptography.X509Certificates.StoreLocation.LocalMachine,
          System.Security.Cryptography.X509Certificates.StoreName.TrustedPeople,
          System.Security.Cryptography.X509Certificates.X509FindType.FindBySubjectName, "localhost");
   
       echo.ClientCredentials.ClientCertificate.SetCertificate(
           System.Security.Cryptography.X509Certificates.StoreLocation.LocalMachine,
           System.Security.Cryptography.X509Certificates.StoreName.TrustedPeople,
           System.Security.Cryptography.X509Certificates.X509FindType.FindBySubjectName, "mycert");
   
       System.Console.WriteLine(echo.echoString("hello"));
       System.Console.ReadLine();
   }
   

   Note that we set the correct service and client certificates. As you can see I’m using an HTTP endpoint. This is useful to test the messages going through a tool like TCPMon. When you have the scenario working you can just switch to the HTTPS endpoint.

Useful references,

• Various ways to create private/public keypairs – http://code.google.com/apis/apps/articles/sso-keygen.html
• Makecert reference – http://msdn.microsoft.com/en-us/library/bfsktky3%28v=VS.100%29.aspx
• SecurityBindingElement Authentication Modes – http://msdn.microsoft.com/en-us/library/aa751836.aspx

1. Java Paas : Building your first app
2. Java PaaS : Handling authentication in your web apps

Before proceeding further let me clarify some terms that we’ll be using . It has become a bit cloudy (no pun intended) since many people mean different things for the same word.

Multitenancy – In the context of Stratos, I’m using the word multitenancy to mean as defined in Wikipedia.

Multitenancy refers to a principle in software architecture where a single instance of the software runs on a server, serving multiple client organizations (tenants).

When you register and get an account in Stratos, that’s a tenant. An organization. Your data will be isolated from other tenants/organizations in the system. When you sign up, you’ll be creating the administrator account for your tenant. Which we naturally refer to as the tenant admin. Tenant admin can add users to your domain, turn off/on various Stratos services such as the ESB, App Server, Data Services Server etc… Monitor data and bandwidth usage of your users, grant/revoke permissions to users.

Let’s me take a little step back here. When you talk about Stratos, there are two major deployment scenarios. One that’s hosted by WSO2. That’s available to users as a service, you sign up, you get an account, your data is hosted on hardware that we manage. This version is called StratosLive. Other scenario, you setup Stratos on your hardware in your private data center, you control everything it’s not accessible to anyone else.

2 scenarios. StratosLive vs Stratos on your private cloud.

Now, coming back to multitenancy, there’s one user we called the super tenant. This acts as an admin account for all the tenants in Stratos. It’s configured at the time of setting up Stratos. In StratosLive scenario, super tenant is WSO2. In the other scenario, super tenant is you. You can do stuff which will affect to all the tenants across the system. Why would you need this? It’s necessary when you’re developing SaaS applications. You’re providing a service which all the tenants can use.

In Stratos, to make your applications SaaSify you need to add a context parameter to your deployment descriptor.

<context-param>
    <param-name>carbon.enable.saas</param-name>
    <param-value>true</param-value>
</context-param>

Authentication mechanism will be the standard form based authentication that we configured in the Jwitter app we developed earlier. This however, does not mean that this is the only thing that you need to build SaaS applications, nor does it allow you to take any existing web app out their and magically turn it to a multitenanted app. For that you need to have a multitenant data architecture for you application.

When you add the context parameter and deploy the web application as the super tenant, Stratos will authenticate against any user account in the system that has sufficient permissions. When you call request.getPrincipal().getName(), user account will appear in the format foo@example.com. From this point on it’s upto you to move to the bits of your application.

In WSO2 data center, we run the entire platform on UEC which ships Eucalyptus and we haven’t seen any of the problems we faced with EC2. Ok, private cloud – very important.

As a platform as a service, it’s important to be able to run on top of many IaaS vendors. Same goes for private cloud solutions. So, if Stratos is to be run on OpenStack the auto scaling code should be working. Stratos auto scales on Amazon EC2 that’s the API that’s been implemented. It’s always a pain to support a different API. OpenStack is said to have an EC2 SOAP API but I couldn’t find the extent to which operations are supported. I’ll definitely have a definitive answer if I contact one of the devs. This post however was not about that.

Mark Shuttleworth wrote an excellent blog about some concerns over OpenStack. As Mark correctly observes, it’s not the right thing to innovate at the API level. As eternal perfectionists it’s very easy to get into having cleaner and powerful APIs. Then again from a pragmatic perspective it’s energy that’s better put into where it matters.

WSO2 Stratos already have a working implementation that is tested. Effort of implementing and testing for another API is going to be time consuming. Whether it’s a bad or a good API when we have to get it running we’ll implement it nevertheless.

There are many concerns that needs more attention than having a good API. Higher IO and network throughput comes to the top of the list ‘cos that’s what it matters from a user’s perspective. It’s great that people debating this issue and looking to make a difference!

As a result of Amazon EC2 issues, mainly relating to unacceptable I/O and network delays hosted version of Stratos which we call StratosLive was so damn slow. Now it’s hosted on real hardware and the site is way way faster!

In the long run, enterprise-wide software are better off running on real hardware. Investing in more powerful machines. Or horizontally scale with a number of cheap machines. I can see Amazon’s appeal for a very short term computing problems. Or as Azeez mentioned, cloud bursting for one. Problems like this infamous article about Amazon EC2 trying to solve. Value you gain for paying exuberant bills for Amazon EC2 is very little at the end of the month. Specially when you’re running it long term.

IMO, this is one reason why a reliable private cloud infrastructure matters. In my experience open source private cloud software has a lot to improve. The free stuff by VMware is good for playing around. When you need I/O and network performance + provisioning, you have to pay big bucks.

• Using WSO2 ESB with SAP ERP (Retail) by Harsha – John Keels Holding PLC is one of the largest conglomerate in Sri Lanka. As a public liability company it’s really interesting how they went about adopting open source middleware products into their core business processes. As the person who spearheaded the effort and provided guidance for the team, Harsha’s talk is going to be interesting providing practical insights into the whole project
• Building Cool Applications with WSO2 StratosLive by Shankar – Stratos provide a rich framework for building applications. Developers can concentrate on making cool applications without having to worry about a whole heap of infrastructure related aspects. They’re taken care of. Interesting to get a head start on application development from Shankar
• Open Source Adoption in the Enterprise by Prajod – Enterprises have been a big no no for open source software. However, it’s been changing really fast. Going to be interesting hearing more insights into open source adoption from Prajod
• Quality – The key to successful SOA by Charitha – Traditional software QA and testing what I read during my undergrad is from a different era. The projects those days usually ran for years. Now, project durations are measured in weeks. Project life cycles are getting shorter and you have to adjust testing and QA efforts accordingly. Have to adopt new methods. SOA testing is even tougher given plethora of possible integration scenarios with other products. Going to be great hearing QA and testing insights from Charitha
• Develop, Test and Deploy your SOA Application through a Single Platform by Chathuri – I’ll confess. I’m not an ardent fan of Eclipse. When you compare it with IntelliJ IDEA, Eclipse does feels like some one kicked you in the nuts. That’s just me I am thrilled to get to know more about Eclipse + Carbon Studio for application development for the WSO2 middleware platform

Full agenda.

<login-config>
    <auth-method>FORM</auth-method>
    <realm-name>Jwitter Auth</realm-name>

    <form-login-config>
        <form-login-page>/profile.jsp</form-login-page>
        <form-error-page>/error.jsp</form-error-page>
    </form-login-config>
</login-config>

<security-role>
    <role-name>everyone</role-name>
</security-role>

<security-role>
    <role-name>admin</role-name>
</security-role>

<security-constraint>
    <display-name>Jwitter Security Constraint</display-name>

    <web-resource-collection>
        <web-resource-name>Protected Area</web-resource-name>
        <url-pattern>/profile.jsp</url-pattern>
    </web-resource-collection>

    <auth-constraint>
        <role-name>admin</role-name>
    </auth-constraint>
</security-constraint>

After this, your application is tied to the authentication scheme Stratos provides. This allows you to authenticate against any user in your tenant. In other words, any user you add to your domain after logging in as admin will be able to login. As you probably guess a tenant and a domain are synonymous in this context. A user inside another domain will not be able to login to your application when you configure it like this.

P.S: Use a browser that supports HTML5.

Instant-On Enterprise concept by HP is brilliant and started in the right time and is heading toward the right direction. Traditional enterprise are seen as slow behemoth monsters that are,

1. Slow to respond to customer needs
2. Slow to respond to current market needs
3. Slow to respond to changing business requirements
4. Have to go through layers of bureaucracy for doing trivial things

… and I bet you can fill a lengthy and exhaustive list with a short amount of time. Things that everyone come to know and taken to be facts of life during their 10 – 15 year tenure. Often these are credited to necessary operational overhead for doing something. Selling a product, a step in the manufacturing process etc…

With the wide spread of Internet, the younger generation with very low attention span- Gen Xers if you will, have grown in numbers. It has become so large that traditional enterprise cannot ignore it. The amount of sales/money/customers they loose every time due to operational overhead is simply not acceptable anymore.

What is an Instant-On Enterprise?

So how can enterprises leverage advancements of technology to make their business processes, IT infrastructure, operational costs for optimal levels? As I see it, this is where HP has defined an elaborate vision with the Instant-On Enterprise. Let me quote the five critical success factors that the Instant-On Enterprise defines,

Jumping the curve

There are a number of barriers that you have to consider in depth when trying to choose a path for making your business processes efficient. The key decision is to choose the right technology. Choosing the right technology is not an easy task. Consider the following points before you make an enterprise wide technology decision,

1. Based on open source and open standards. Open source is not hobbyists for anymore. A large number of critical business functions runs on top of open source software. When making a choice you have to make sure the stability and adoption of a particular piece of software. There are many open source alternatives for any given problem. Choosing open source alternatives is not simple anymore.
2. Business friendly license. Make sure to choose software components that have business friendly open source license. Where if necessary you could build upon and sell your solutions without having to reveal the source. If you don’t have such requirements, great!
3. Flexibility and performance. The solution that you choose should be able to configure to your business requirements without having to write copious amounts of custom code. Having to make changes to the core of the software in order to support your simple requirements is cumbersome, hard and time consuming. Time you can use to improve your business. As your business grows, software components that you choose should be able to handle demanding business needs. Shouldn’t disrupt your day to day operations while handling demand spikes and general increase of traffic.
4. Ability to use only those features that you need. No more. No less. Even though you have infinite resources (thanks to Amazon for example) at your disposal doesn’t mean that it’s ok to run bloated software with several hundred features that you’ll never need. You should be able to run with only the features that are absolutely needed for your scenario and discard/turn off all other features.

How to implement an Instant-On Enterprise?

Did the last section sound as if it’s a pipe dream? Couple of years ago it would have been, yes. Let me show you how you would go about implementing an Instant-On Enterprise. Infrastructure as a Service providers like Amazon EC2 are too low level when it comes to having an Instant-On Enterprise. Yes, it’s a critical part of this solution but you need a high level platform which operate on top of this elastic infrastructure.

WSO2 Stratos is built to answer exactly that. Hosted version of WSO2 Stratos is called StratosLive. Most of the corporates doesn’t like their private information lying around in public servers. If it’s the case, you can download Stratos and host it inside your corporate data centers.

Now let me revisit HP’s critical success factors again and show you how Stratos helps to achieve those.

Tools needed for building an Instant-On Enterprise is right there. StratosLive has a free plan for you to play around and make yourself comfortable. Also has paid plans with differing SLAs for serious business usages. You don’t have to spend your time and money to buy some fairy tale big vendors usually preach about. You can play around with the entire platform and start building the bits and pieces you need for your enterprise. For free!

After Jonathan’s keynote, I went for the CloudSandra presentation where they’ve implemented a framework with Brisk. The presentation seem to be intriguing as they’ve developed a multi-tenant REST API for Cassandra. Although the meaning of multi-tenancy seemed to have interpreted differently as I’ve been used to the term. Multi-tenancy as I know it have been used to treat a particular organization (an entity as you will) as a tenant and you can have individual users inside that tenant. I felt the multi-tenancy in CloudSandra meant more like multi-user system. I may be wrong. Their usage of Apigee to play with the REST API was very cool as Apigee provide a neat UI for your REST APIs.

Next was an entertaining talk by Eric Evans. Was quite fun as Eric the guy who helped popularize the term NoSQL and the talk was regarding SQL like query language for Cassandra called CQL! It was mostly about high level info/rationale behind coming up with CQL and the current state and some hints as to how it might evolve in time to come.

After that there was a session about Brisk. An introduction and what Brisk is all about by Jake Luciani.

Then David Strauss did an entertaining talk where he demoed a highly available DNS server written using Cassandra. He showed how data propagetes to 3 machines hosted in 3 geographical areas. Very cool!

During Adrian Cockcroft‘s presentation he described how Netflix is using Cassandra, how they migrated to Cassandra from on Oracle DB. Also plethora of other operation aspects of Cassandra clusters. One point he stressed during the talk was, since they were running exclusively on EC2 nodes they had to take instance termination as a fact of life. So they write apps so that they’re resilient even couple of nodes go down. They also deliberately kill random instances and test if the system function as expected!

As the final session of the day attended the Cassandra internals by Gary Dusbabek walked the audience through the Cassandra code base. How everything is wired together and where critical components are at.