Tuesday, January 29, 2013

Why we built a Raspberry Pi cluster

My previous blog post on how we built a Raspberry Pi cluster became an instant hit within a few hours of publishing it. Sites such as ycombinator, reddit & it was even trending on Google+. There were many positive reviews & comments, and there were a few critics as well. Most of the critics argued that the Raspberry Pi is a very slow low end platform, and for the time, effort & money spent on such a project, they could easily purchase a powerful core i7 machine & achieve the same or better performance. Some even argued that it is much cheaper to use Amazon EC2.

The critics

Most of these critics seem to be missing some key points. In general, the plug & play culture promoted by large companies seems to have killed the creativeness of most individuals to the extent that they cannot even admire or appreciate such a project. I even found a hilarious post titled "Getting Started with Raspberry Pi Is not Easy as Pie". The Raspberry Pi is not for the faint hearted, lazy people who expect things to work automagically when plugged in. It is for the hobbyist, enthusiast & creative minds, and provides a platform to bring out the creative genius in those individuals.  Most people are very reluctant to move out of their comfort zone & try out new & challenging stuff that might lead to failure. If something looks even remotely impossible, they are quick to turn away from it, and worse, discourage others from trying things out. 

The creative genius

Having had the opportunity to work at WSO2, where people are encouraged to be creative, try out new things, fail from time to time, pick up themselves & try again, and where failure after trying something new is not frowned upon, we are more open minded & are comfortable at regularly stepping out of our comfort zone, trying new things, and getting to the next level. After all, if you are not failing from time to time, it means you are not taking enough risk & trying out new things.

Why we did it

So let me list down what we've achieved by building this Raspberry Pi cluster;

0. Nobody has done it! This is something we are quite familiar with. When Gartner predicted that it will be 2015 before a full cloud enabled application platform is available in 2010, we already had built a major part of it, and had all the core components.

1. We have demonstrated that enterprise grade middleware, does not necessarily have to be bulky, heavy weight & require gigantic amounts of memory & processing power. In fact, we have a few customers who are running & who are planning to run our middleware on resource constrained environments. By demonstrating that we can have good performance on a platform such as the Raspberry Pi, we prove in the real world what our software & the team at WSO2 is capable of.

2. Provide an opportunity to the team to bring out their creative genius. This directly contributes to highly motivating the team & providing the necessary stimulus to bring out more creativity.We are involved in the business of solving problems. Solving problems requires motivated teams & creative genius.

3. Encourage the team to venture into areas outside their normal day to day office work & stimulate the creative areas in their brains

4. Promote the culture of creativeness & taking up challenges, to people outside WSO2, thereby helping others to reach the next level. There are too many negative minded people out there who are quick to point out that things cannot be done, or that doing things in a certain way will lead to failure. We wish to do our part to counter the effects of such thoughts & promote creativity & encourage others to try out new things & venture into unchartered territory.

5. The Raspberry Pi project is a new venture, and one way in which it would improve is by people adopting it & trying out new things with it. These efforts will lead to the improvement of such projects which have been started off with noble intents. It is only going to improve, and when it improves, we & others will also benefit from it. 

6. We have experience running large scale systems on real hardware as well as IaaS such as EC2, and know what the cost benefit tradeoffs are. It is left as an exercise to the critics & readers to calculate how much it would cost to run a medium size production  cluster on Amazon EC2 for a year.

Constructive criticism & ideas are always welcome!

Post a Comment