This phrase is very true in the case of cloud. So long when architects design an application they always think about the resource availability in a given (hosted server or a cluster of servers) scaled out scenario. Cloud totally breaks this paradigm and allows one to think about server itself as an available resource in the environment. Let us consider an example of a batch payroll process job. How do we architect that in a normal scenario?

  1. You queue the jobs
  2. A set of servers dedicated for that job picks it up and processes it
  3. Within a server we could spin of multiple threads based on the available cores to process it in parallel

Now assume there are 5000 jobs waiting to be processed in the queue and the accepted latency between in to out is 60 seconds. If one job takes 1 second to process, even if we run multiple threads we would need more than 5 servers to process them. Added to this, if this job processing needs to happen only in a specified period of time of a day we would simply be idling out these servers rest of the time. This kind of scenario exists in almost all applications. The applications simply either compromise on the latency or bear the cost of the idle server time. So, how does this architecture change with cloud?

  1. You queue the jobs
  2. Have one server look for the number of jobs. Based  on the volume get a server on the fly with a predefined setup and ask the server to process few jobs
  3. Each of these servers can also run multiple threads.
  4. Once a obtained server finishes processing, stop and release the server instance.

Does it not sound simple like a memory allocation and de-allocation?  As for the technical complexity of achieving this, with existing technologies we know that creating a thread in a program is as simple as creating another object instance. Similarly now with the leading cloud providers like Amazon EC2 creating a server is nothing but creating another object instance (well, it is a little more complex than it sounds but definitely possible with a proper architecture). This does save a lot of cost and enable you to effectively use the resources.

Thanks to the cloud which gives the architects the freedom to think beyond servers for a cost effective solution.