What is cloud computing?

According to the definition in Wikipedia .. “Cloud computing refers to the provision of computational resources on demand via a computer network. 

Cloud computing can be compared to the supply of electricity and gas, or the provision of telephone, television and postal services. All of these services are presented to the users in a simple way that is easy to understand without the users needing to know how the services are provided. This simplified view is called an abstraction. Similarly, cloud computing offers computer application developers and users an abstract view of services that simplifies and ignores much of the details and inner workings. A provider’s offering of abstracted Internet services is often called “The Cloud”.

For the purpose of this post, I am going to focus on the Software as a Service (SaaS) type of cloud computing which delivers a single application through the browser to thousands of customers using a multitenant architecture.     Salesforce.com is by far the best-known example among enterprise applications, along with Flickr, Google Apps like Blogger and Zoho Office.

What does this mean for Quality Assurance planning of a SaaS cloud application?

Similar to a hosted environment, the cloud computing now takes on the role of the customer’s IT infrastructure that was previously managed in-house.  The cloud application provider must inform and commit to customer a high standard of quality in the following areas:

  1. Release Management
  2. Compliance and Security Certification (i.e. PCI, Sarbanes-Oxley)
  3.  Performance / SLA monitoring

Release Management in a Cloud Environment

The SQA role in Release Management in any hosted environment does not stop once the software has been approved for release.  The full deployment cycle needs to be carefully managed to have a seamless experience for the customer with little downtime or outages.

1. Multi-platform Support – because browser applications are the point of access for the client to the cloud application, browser and operating system compatibility  issues are a critical concern.  Given the wide range of customers to be supported, there is demand to support a wider range of platforms and a greater business risk if defects are found in this area.

  • Supported browsers should be determined based on their compliance compatibility.  Eliminate any older browser or operating system versions that will cause issues with compliance or coding due to non-standard web support (see http://www.webdevout.net/browser-support-summary) and/or security issues.  This will make QA and support much more manageable.  This list should be reviewed and updated regularly.
  • Customers should be clearly informed of the browsers that will be supported as well as any browser configuration requirements.  It amazes me how many clients stay with ancient browser versions and expect them to be supported!  This sounds like an obvious point but it often is overlooked by application providers and causes a great deal frustration for customer support when it isn’t clearly understood.
  • Constantly monitor browser and operating system security alerts, hot fixes and new releases to ensure that the cloud application is always compliant and defect free.  Always check the major platform release schedules when determining the cloud application release schedule.
  • Block unsupported browsers where ever possible.  If the application can detect and block unsupported browser versions, this will go a long way to reduce the potential of unexpected errors and corrupted data .
  • Automate cross browser testing.  There are several excellent tools in the market now to ease the burden of cross browser testing.   (see http://freelancefolder.com/7-fresh-and-simple-ways-to-test-cross-browser-compatibility/ ) Automated API tests are more economical for testing the business logic layer of the application and the cross browser testing should focus primarily on the GUI issues of the application.

2. Customizations

  • As much as we want to always meet our customer’s needs, customized release versions can throw a real monkey wrench into release deployments and the cost of software support.   Customizations complicate the version control management of  the source code and each release must be tested against every customization and customization hot fix.  Giving a customer a quick customization might sound like an easy fix from a development perspective but it quickly becomes a release management and QA nightmare to maintain and support. Where ever possible, a better solution would be to deliver customizations as a configuration option in the core product.

3. User Acceptance Testing and Training

  • Prior to deploying a major release of the cloud application, the customer should have access to the new version in order to assess how any changes might impact their business workflow and processes.  It also gives customers an opportunity to participate in the user acceptance testing (UAT) efforts and an environment to provide early training for their teams prior to the deployment.

4. Data Migration

  • Many cloud applications offer migration scripts to assist new customers in migrating their data from a legacy system to the cloud application which enables new customers to have a quicker conversion and ramp up time.  Sometimes these migration scripts are a standard component of the product and thus, part of the QA scope for testing.  However in some instances, these migrations scripts could be a customization requiring only one time testing.
  • If a new release requires data migration for existing customer upgrades, it would be ideal to offer the customer a beta environment with their own migrated data to better assess any data issues or impacts.  This also allow the IT team a chance to test out the migration on customer data prior to the release.

5. Deployment Planning

  • Customers will not necessarily want to accept a new release according to the cloud application’s release schedule.  Therefore the cloud application should provide a process for the customer to remain on their current release and to choose when to upgrade according to their business demands.  This also eases the burden on the cloud application IT team to not have to migrate every tenant in the application at once.
  • Many cloud applications have service level agreements (SLAs)  for 7-24 availability.   In one company I managed the staged deployment to a clustered server arrangement where we brough one side off-line for deployment for a short period of time while the other half remained online.   Deployments were planned for off-peak hours to limit the risk of performance issues.  Once one half of the cluster was deployed and tested, it went back online and the second side was taken off-line, deployed and tested.   It was critical to have a detailed deployment plan complete with milestone checkpoints and rollback scripts for each checkpoint, if needed.  We never had to use these rollbacks but they were as well thought out and tested as the successful deployment plan.  Each deployment script had a dress rehearsal in the QA staging environment prior to the deployment date so all team members were familiar with their tasks for the actual deployment.

For more information, see these great blogs on Cloud Computing http://cloudtp.com/thought-leadership/top-25-cloud-bloggers