Cloud Testing : The Cloud and Our Testing Practices

The aim of this study is to explore to prepare software testers for a platform shift. Doing software testing in cloud environments offers economies and scalability possibilities that are intriguing to software development companies and departments. Interest is high because purveyors of "the cloud" promise to reduce software development and testing expenditures while maintaining or increasing the Quality of Service (QoS). Do those claims hold up? That's what we'll explore in this study.


INTRODUCTION
Cloud computing is a new software service solution that holds an entire infrastructure and environment in one location, which is accessible to specified individuals via the Internet.Both hardware and software may be housed in a cloud environment as shown in Fig. 1.
Predecessors to cloud computing include Software as a Service (SAAS), Fig. 2, a service solution which is similar to cloud computing that allows users to access the software/hardware onsite or via an Internet connection.
Service Oriented Architecture (SOA), which could also be accessed either onsite or offsite; and Application Service Providers (ASP), which hosts applications (Xiaoying, 2011).Compared to its predecessors, cloud computing seems to be better positioned in terms of economic viability, cost-effective approaches to scale and reliability, early adoption of interfaces and open source implementations.The objective of this study is to explore the new challenges and opportunities that Cloud computing has opened up for testing departments.

BEING SAFE ABOUT CLOUD TESTING
Since cloud computing systems hold vast amounts of corporate information and can only be accessed through the Internet, the availability of the Internet and the reliability of its connection are two facets of cloud computing that users depend on the most.And, although the software and/or hardware infrastructure is harbored "safely" in a cloud computing environment, the network will not always be trustworthy (Gao, 2011).
Well, it's obvious that there are a number of things that can go wrong, such a weak telecommunications signal, a storm or an Internet provider going out of business.For these reasons, it is a good idea to prepare a backup plan for accessing the Internet.Think about purchasing a secondary connection, in addition to the primary one, that can be available when you need it most.You may even be able to configure the system to cutover or re-route to the secondary line if the primary line experiences too much traffic at one time.If you choose to do this, just be sure that your backup wire is not from the same provider as your primary connection.
The real fact is that computers used in a cloud environment will never be 100% reliable; there will always be viruses and other glitches that may slow down the system.So, even with the best Internet connection, a robust computer that can handle high processor and memory speeds and has a large hard drive space is a must (Vouk, 2008).
Cloud environments cannot be externally controlled by the companies using them, which mean problems with quality are difficult to fix once the cloud is fully integrated with the corporate system.In an ideal situation, a specific environment would be created for testing each application development, configuration management, training, etc., thereby helping to ease change, versioning, release management and identify quality assurance issues.A detailed conversation with your vendor will be handful to find out if replicated environments are offered as a part of the service.In most cases, though, such a luxury typically defeats the purpose of using cloud in the first place, as it is not very cost-effective.It is in your best interest to work with the cloud service provider to construct a quality assurance "staging" area where all of the testing, configurations and setup are finalized prior to going live.You should test during this beginning phase until you're satisfied that you have the best cloud service solution to suit your needs.

KEY BEST PRACTICES FOR CLOUD TESTING
When possible, test the cloud applications in a very similar, or the same, environment to the one in which it will be accessed when it goes live.The testing should scrutinize the application's performance, reliability, speed, security and functionality.Recently, the traditional functional testing, also known as regression testing, is being used more than any other kind to validate cloud; which a one-pronged approach is.To truly ensure an operable cloud environment, performance and reliability tests should take the front seat, during which probes can be used to capture statistical data and report on the consistency of the application.The strength of cloud computing's security barrier for user-protection and corporate compliance is crucial, especially if your company will store sensitive information in the system.Formal security testing tools and even hacking techniques are some of the most effective methods for testing the security of a cloud environment (Gao, 2011).And, a disaster recovery test will help you confirm that the vendor is reliable and responsible when faced with an emergency.
A cloud computing test plan should also be created at this point and should include a detailed log of every single testing activity and issue that arises.Prepare a list of metrics that you want to be reported to the vendor, such as defects or errors found, the speed of service, reliability and so on.These indicators will help top management and staffs address the maturity and consistency of the cloud vendor's processes.If these steps are still not enough to put you at ease, create an audit of the system's log file and reports and prepare an internal and external communications plan with the vendor.Get in the habit of holding at least one weekly meeting during which you discuss future changes, status, metrics and outstanding action items.The more communication you have with the vendor, the better they will understand your priorities and the better service you will receive.
In-house policing of cloud QA: Once your company had decided to use a Cloud service solution, there's still plenty for QA managers and testers to do in house.For instance, make certain that the developers and/or users have what they need to do the task at hand.Naturally, there will be need for planning testing of applications for future projects.
Policing employees' storage habits is a key inhouse project management practice.Developers, testers, managers and users should not continue to save important files on their computer desktops and various locations on the corporate network instead of using the cloud system as the central source.Retrieving data can be a nightmare if you don't have access to a computer that has all the important information on it.Even worse is having to search through lines of folders on a corporate network for a document that someone else authored and saved somewhere (Scott and Tauhida, 2010).
To prevent improper file storage, QA managers may need to mandate that all employees use the cloud architecture when saving files.They could enforce correct file saving by adjusting the read-and writeprivileges on each computer that has access to the cloud system.Another option is using local proprietary lockdown software.For instance, you may allow dataentry personnel to use the Internet for cloud applications, but prohibit them from obtaining writeaccess to their desktop so that data cannot be transferred to that location.Or, cloud users may be prohibited access to public Internet sites, such as email, which could be used to share sensitive corporate information.Another solution may be to use Internet machines with comprehensive capabilities instead of a traditional desktop computer, which could potentially save money and reduce the risk of data misuse (Chan et al., 2009).
In general, be prepared to relinquish a certain amount of control of the system either way.While you must be cautious about getting good service level agreements and preparing well, those are not good reasons to dismiss using cloud services.I suggest getting started with cloud services with software products that are already stable and well-tested.It is also useful for a solution with products that will be developed within a Cloud environment, as long as R&D remains consistent.
Regardless of the reason you choose to use cloud, however, using the techniques discussed above may help you get the most out of the service while remaining cost-and quality-effective (Xiaoying, 2011).

Cloud testing: the cloud and your testing practice:
The impact of the cloud on testing practices has grown with the cloud's growing presence in the IT space.Testing practices are now dealing with several aspects of the cloud simultaneously-three aspects of the cloud that directly impact our testing practices are: using the cloud to create scalable testing environments, nonfunctional testing of cloud-based solutions and functional testing of cloud-based solutions (Xiaoying, 2011).In short, the cloud can be used: • As a testing enabler • For non-functional testing • For functional testing (unit, integration, system and regression) While these are clearly distinct aspects of the cloud space and the discipline of testing, there are relationships between these aspects of the cloud that are being ignored or "glossed over" by both vendors and proponents of cloud-based computing specifically the non-functional risks around security/integrity and performance.

The cloud ASA testing enabler:
The cloud provides the opportunity to create scalable testing environments that can be easily ramped up or down given the immediate needs of your testing organization.Whether this type of scalable solution is an appropriate fit for your organization is dependent on the idle time of your development and testing infrastructure throughout the year.If most of your infrastructure is in use, or your infrastructure is inadequate, then a straight investment comparison of own versus rent/lease should be possible.On the other hand, if much of your development and test infrastructure remains idle throughout the year, then leveraging a cloud-based solution may alleviate your overall infrastructure cost.
There are other factors that must be considered before moving testing assets to a cloud-based solution.These factors do not involve the capacity of cloudbased testing solutions-the capacity is certainly there, but instead the security and integrity of these solutions.The security of cloud-based solutions still remains problematic with major security incidents happening on regular basis (Baskaran et al., 2010).If your production environment has not moved to the cloud and you have not created obfuscated test data, then you are exposing your organization to significant security risk by moving testing to the cloud.The question to ask is, "Am I exposing my organization to additional risks by moving to a cloud-enabled testing solution?"If the answer is "yes," then a proven risk mitigation plan needs to be put in place before moving testing assets to the cloud.

Non-functional testing of cloud-based solutions:
As your IT organization and infrastructure moves into the cloud, non-functional testing becomes critical.Recent experience has shown us that most of the risks associated with the cloud come from non-functional requirements not being met or often not even being articulated and therefore not being tested or supported.The areas of proven vulnerability are performance, security and disaster recovery/management recent examples being: • Amazon's "glitch" (the last week of April) that caused numerous Websites hosts to crash or run very slowly.• Sony of Japan revealed that hackers accessed 100million Play Station accounts including names, addresses, passwords and possibility credit card details.
Amazon Web Services (AWS) Virginia data centers in its US-East-1 region were down leaving many of its customers with no service and no service alternatives as shown in Fig. 3.
From disaster recovery/management perspective you need to ensure a plan has been put in place by the cloud provider to ensure interruptions in their service will be addressed.This is somewhat problematic since the key cloud providers have not yet addressed this issue witness Amazon's "glitch" in late April.Your testing organization still needs to identify the risk and test any recovery procedures presented by the cloud provider.
From a performance testing perspective the testing organization can apply traditional tools and techniques while ensuring the infrastructure of the cloud-based solution closely resembles (or is) production.There are additional factors that will need to be addressed, the most critical being: • Addressing the loads that will be applied by other clients/customers of the cloud provider.• Addressing the loads that will be applied against the internet providers (example: Cyber Monday).
From a security testing perspective, the testing practice will need to become much more aggressive than many test organizations have been in the past.Your business and your data now reside on a third party that has not yet presented a sound security solution -with all transactions travelling over the Internet.The testing organization will have to address the security of the application presentation layer, the service layer, the data layer and now the architecture/infrastructure to ensure security requirements have been met.In the past, security risks have often been mitigated by the nature of the architecture-in-house applications on a secure network with little direct contact with the "world."Now Fig. 3: Amazon Web Services (AWS) (gigavox) your applications will exist in the cloud, or cyberspace, with all the benefits and risks that provides.

Functional testing of cloud-based solutions:
The test processes and technologies used to perform functional testing against cloud-based applications are not significantly different than traditional in-house applications.Adjustments do need to be made for the non-functional aspects of the application space that relate to security and data integrity (Xiaoying, 2011).If testing involves production data then appropriate security and data integrity processes and procedures need to be in place and validated before functional testing begins.

CONCLUSION
Awareness of the non-functional risks around the cloud are critical to successfully testing (functional and non-functional) or leveraging (test labs) the cloud.The responsibility for testing the non-functional aspects of a cloud-based application may reside within the testing practice or the infrastructure/security team (Fang and Xiong, 2011).As a rule of thumb, if you are dealing with a public cloud or a provider-supplied cloud, then testing responsibility should stay within the testing practice.If you are dealing with a private cloud, then testing responsibility could be handled by the infrastructure/security team-at least from a performance and security perspective.In either case, all parts of the IT organization need to work together to mitigate the risks presented by cloud-based solutions to the overall enterprise.