Here’s an interesting concept I hadn’t seen before: “acqui-hire” or “acqhire”. (yes.. far-fetched indeed)
I saw articles of both Facebook and Twitter acquiring companies not just for the technology, but more for the people. Read here (Facebook-Gowalla) and here (Twitter-Whisper Systems).
It’s no coincidence that both these companies have chosen to use a commodity data store (MySQL in both cases), and created top notch teams around it to sure performance. In addition, have a look at Twitter’s Gizzard. It’s a middleware solution that sits in front of the databases and deals with sharding and balancing. It’s written in Scala by the Twitter team: this is what you can do with a great team!
Interesting point to note: Data writes through Gizzard should be idempotent AND commutative. Practically this means that a write can be retried later, without a major impact on consistency (resulting in eventual consistency). The two constraints should be governed during data design. The team calls them easy to adhere to, if you start off with large scale in mind, and adjust other areas of your code.
Monday, December 12, 2011
Thursday, November 24, 2011
While my friends in the US are enjoying their lovely Turkey/Thanksgiving dinner, I have made myself useful today and created an overview of a typical cloud stack evolution. What better case study to use then Amazon Web Services?
|Amazon Web Services evolution|
Let's walk through the evolution step by step:
- AWS started with the basic building blocks used by applications: compute (EC2), storage (S3) and network/communication (SQS).
- Interestingly, the next step is really to enable monetizing the solution. Amazon is of course already making money from the basic services. However FPS (Flexible Payments) provide an incentive for 3rd parties to contribute machine images (AMI's) into the ecosystem, and make some money from them. I believe this has been one of the motivating factors for companies with commercial solutions (think Oracle, IBM) to get on-board.
- Climbing up the stack, an alternative data or storage service is added: Simple DB. This a great alternative for S3 if you're not storing multi-megabyte files.
- Elastic IP Addresses, Elastic Block Storage and CloudFront are added to optimize the current offerings. Better accessibility from the Internet, more manageable storage mechanisms for host instances and optimized content serving.
- By this time, Google released its Google AppEngine (for python). Developers with zero knowledge of infrastructure components can now quickly deploy applications ("in the cloud" ;-)
- The next set of AWS services are really getting the environment ready to move into the PaaS arena: more automation and more dynamics.
- CloudWatch provides insight into the infrastructure and allows us to take action based on metrics
- Elastic Load Balancer integrates and automates the load balancing process, which developers previously had to run within a virtual machine (using Apache/Nginx/..)
- Auto Scaling allows for auto-provisioning of components based on certain metrics.
- Next up is a service I'm sure lots of enterprises had been asking for: Virtual Private Cloud. This will allow you to hook up your private data center with AWS services, effectively creating a hybrid infrastructure
- Relational database services get added
- A major step towards a complete application platform is CloudFormation. It's now possible to define an entire application environment and have it be automatically provisioned as one logical service
- DNS services are added through Route 53, which provides a convenient way to manage domain names and how they're mapped to IP addresses (or applications)
- That last feature can almost be considered the last mile before we can get to BeanStalk. With BeanStalk developers can literally provide a deployment unit and the service will spit out a URL at which your application is available. There's of course a huge amount of magic happening behind the scenes.
I took Amazon 5 year to deliver PaaS, and they've been leading the way for most of that time. So I cringe when I hear people talk about PaaS as if it's something that can be unboxed, installed and be done with it.