A matured SaaS solution is one which can scale as and when it is required without touching or modifying the underlying source code. So, how to build a scalable Web based SaaS application? A general myth about Scalability and cloud is that, the application is automatically scalable if it is hosted on Azure, Amazon or any another cloud provider for that matter which is false. The real fact is that they offer commodity hardware or software/platform to develop and deploy application into multiple virtual servers and provides indefinite elastic Scaling, but the real truth is that the application has to be built layered, SOA/loosely coupled, modularized, componentized in order to be eligible for scalability.

On a high level, there are two broader ways of Scaling, they are 1. Vertical Scaling 2. Horizontal Scaling, let’s forget about vertical scaling at this moment because it is all about increasing the power of the single instance/machine by adding additional CPUs/RAM/Hard Disk etc and make the system to handle more load.  Horizontal Scaling on the other hand is the process of replicating the application in a web form and introduce a load balancer to route the request and response of the user and thus make the application respond faster by dividing the load among multiple nodes.

Deciding on whether to adopt Scale up or Scale out strategy depends upon on various factors including Cost, Reliability, Availability and management etc. It is always wise to decide the Scalability decision upfront before putting together the design or architecture, because the application architecture will vary between Scale up  and Scale out strategy, if Scale up is chosen, the architecture could embrace techniques such parallel processing or multi threading to utilize the maximum cores available within the system, where as in the Scale Out model, it is difficult or complex to use threading  techniques to complete a job because the work load is distributed among multiple nodes.  It is also possible to combine both Scales up and Scale Out capabilities, provided the application is equipped to handle both scenarios.

Drawbacks of Vertical Scaling

1. Becomes a single point of Failure
2. Costly
3. Scalability Limit
4. Difficulty in Maintenance/Enhancement/Bug Fix etc

Benefits of Horizontal Scaling

1. Difficulty in Maintenance/Enhancement/Bug Fix etc
2. Cheap/Cost Effective
3. Elastic Scalability
4. Pay as you Go

Design for Scalability

All the components and functionalities of the application has to be developed in a loosely coupled model by having interface layers between components and have consistent messaging mechanism to inter operate, do not mix and match different messaging mechanism as you may need to boxing and un-boxing which might attract some performance downgrades. Intelligent usage of Distributed caching can help avoid costly round trips to databases which can significantly increase the performance of the application.

It is always advisable to run the jobs/task in the back ground rather than trying to do everything upfront in the same process, adopting queue mechanism can help in queuing up all the requests that comes from the front end and make it a queue and use worker roles or compute processors to poll these queue and complete the job in parallel.

Please refer here to second part of this article