“Like all 2-year projects, they fail” – This came from Dr Werner Vogels VP & CTO Amazon.com (who is actually a very very good speaker) while I was attending a free promo afternoon for Amazon web services. I think this endorsement of JFDI as a methdology is both wonderful and frightening to hear, frightening as I’m sitting next to Ian ‘the Gremlin’ Watkins, who practically personifies the act of just f**king do it (his little eyes sparked and I could see a ‘fag packet’ tech spec forming in his mind).
But I digress, as expected from Amazon it was a very good sales pitch and had the technical proof behind it to satisfy the corporate techs that filled the audience. Such an example was Nasdaq who built a full reporting system off all the Wall Street historical data by just pumping the raw Wall Street data into text files (actually it was a split on their current data pump from Wall Street) stuffing them onto Amazon S3 then hiring an Adobe Air dev to do a front end, thus producing a global scaling application for the cost of $30,000 Dev work and $500 Amazon storage cost, no screwing around, no extra stuff to buy, Oo er!!!
To put it at its very crudest, AWS comprises of virtual servers (they call them instances) which are called “EC2”, virtual storage of various different types (S3 being the one you have heard of) and the normal batch of peripherals services done as API (such as email, e-commerse, monitoring and the like) if you want more details go to http://aws.amazon.com as they explain better. However, where it starts to differ from what we are used to, is in the way these services are available to you without the huge initial infrastructure costs. Also, upgrading is a smooth and swift affair rather than the jerks that are normally associated with adding new servers (they gave a good example of a website that became an Internet meme due to Facebook and whose instance/server numbers went from 100 to 3500 in one weekend because of the demand, something quite impossible in a normal infrastructure setup)
Dr Werner stated that at the core of cloud computing are 2 things, “on demand” and “pay as you go”. If it’s missing these it’s not cloud computing!!!
‘On demand’ = Needs to grow and shrink in an unlimited automated fashion without manual intervention.
‘Pay as you go’ = A non-contract, ‘only pay for what you want’ model, ‘if you don’t like our service, you can leave whenever you want’
Amazon were at some pains to point out that Amazon.com does not get any preferential treatment on the amazon cloud (listen to this, bloody Apple!!!) so fair are they that they even host Netflix who is their direct competitor in the US. They also say that they treat Amazon.com and Amazon AWS as separate companies with no knock effect, i.e. when the amazon website gets busy in the xmas rush, they ‘wont pull the rug, out from under our AWS clients’ (this statement seems to have been born out of some FUD that their competitors have been spreading around).
For those of you that already use it in some way (say for jungle disk backup) the reason it’s so bloody cheap is the economies of scale and believe me, they are huge, they purchase ’10s of racks each week’ of servers, which apparently lets them get away with murder with the hardware people. This led on to a rant about vendors who say “no no no, you don’t want to use Amazon cloud, what you want is to build your own cloud…..with our kit”. He then introduced VPC (Virtual Private Cloud), so you can test a walled-off part of the cloud to see what it’s like with no risk. Oh, on that note they seem to be borderline obsessed with security, first port of call on that is their ‘Security Best Practices’ white paper found at http://aws.amazon.com/security/
A few nice touches to it are:
1) They allow you to place your data in certain regions for jurisdiction reasons, e.g. EU data wont ever be stored in the US if you don’t want it to.
2) ‘We don’t want to tell you what tools to use to build your apps, build them with what you want’ (you still listening Apple)
3) They have these things called “spot instances” in which you can state how much a job or task is worth, and Amazon will only run it if the cost is low enough, say for data mining for reports (the reason for the fluctuation in cost is based on how busy the cloud is and therefore how much left over processor time they have lying around).
4) You can remap an IP address from instances (servers) to other servers via API calls and have it take full effect in seconds. They call this ‘elastic IPs’ (well I think its cool).
5) Packet sniffing does not work, your instance (server), can only see packets destined for it (promiscuous mode or not) – this does not apply to VPC if you have it.
They did give a example of scaling: there are 120 billions S3 objects and when load testing the US turbo Tax application they reached 300,000 concurrent submits before deciding that they weren’t going to break it.
Downsides:
1) Some of the clients had the problem of getting finance to understand the change between capital versus operating costs.
2) As cost is done on demand, the naffer and more inefficient your code/app is in terms of network and CPU use, the more it’s gonna cost you. Finance will see this and nail you to the wall sooner or later (well actually this is a good thing really but only for good developers, alas I’m not such a one {just ask my clients}).
3) It was noted that support from the SME level of privider was limited, perhaps we can all bone up on this to fill that gap, eh?
All in all a very exciting afternoon, and one that has left me with the desire to do a number of projects/proof of concepts (and they gave away free beer). 🙂
