Cloud-based software testing good but unnecessary

Cloud-based software testing good but unnecessary

Summary: Testing software, particularly mobile apps, in the cloud has cost advantages, but companies should balance this benefit with security and data compliance requirements.


Cloud-based testing of software applications brings cost benefits but this alone does not make it a top priority for most companies as they still have to contend with issues such as data security and interoperability of cloud systems.

Chandranshu Singh, senior analyst of IT solutions at Ovum, said the volume of software testing companies have to conduct has gone up tremendously in today's business climate which increasing relies on software to provide a competitive edge. So the most obvious advantage of cloud-based testing tools is that these bring cost effectiveness as there is better utilization of existing infrastructure, he noted.

Other benefits would include time savings in the setting up and tearing down of physical test laboratories, thanks to the preconfigured environment templates and instantaneous resource provisioning which allows IT staff to scale their tests on demand, Singh added.

Performance testing is particularly well-suited for the cloud, as load simulation can be done easily in the cloud, he pointed out.

Jeff Findlay, solution architect for Asia-Pacific and Japan at Micro Focus, added organizations also get the assurance that a huge volume of traffic will not dent the performance of a software. This would have led to much user frustration in the real world, he noted.

Agreeing with Singh, Findlay said cloud-based testing also eliminates the need to build and maintain required infrastructure for peak load simulations, making it far more convenient and cost-effective.

Not a compulsory capability
Compelling as cloud-based software testing tools can be, these are not an enterprise necessity right now, both observers stated.

Singh said most inefficiencies that plague software testing teams can be handled by first finding the proper balance between automated and manual testing. So rather than basing the decision on how to conduct the test, the better question would be "under what circumstances would cloud-based testing benefit my development effort?"

The Ovum analyst said another issue is the low level of interoperability between different clouds. He urged organizations to take that into consideration when selecting vendors, to ensure that migration to a different cloud platform for testing will not be a major problem should the need arise.

Furthermore, not every public-facing application is a viable candidate for cloud-based testing, Findlay stated. For example, certain apps may have industry-mandated security constraints, so the software and data's test conditions require a more tightly-controlled environment than online, he explained.

Data security and integrity in the cloud remain contentious issues to date. While vendors are making strides to address this, it is essential for organizations to keep control of their data and administer the structure of that data in accordance with their quality processes, the Micro Focus executive said.

Mobile apps key niche
Mobile developers and companies building their own mobile apps should utilize the benefits of cloud-based testing though, Singh said, adding it "makes a lot of sense" for such software.

This is because when it comes to testing mobile apps, the device and platform are key considerations but the fragmented market of mobile device operating systems makes it difficult for companies to test their apps for all platforms and devices, he explained.

Thus, engaging a software testing provider that offers access to the different platforms on a subscription basis saves times and costs, making it a "no-brainer", he noted.

That said, Findlay called on companies to be aware of the environments their mobile apps are being tested on, which could impact the accuracy of test results. Many mobile testing-as-a-service providers utilize rooted or jailbroken devices as test platforms. Apart from security issues, these test environments do not reflect the "real world" and could skew results, he cautioned.

Topics: Cloud, Enterprise Software, Software Development

Jamie Yap

About Jamie Yap

Jamie writes about technology, business and the most obvious intersection of the two that is software. Other variegated topics include--in one form or other--cloud, Web 2.0, apps, data, analytics, mobile, services, and the three Es: enterprises, executives and entrepreneurs. In a previous life, she was a writer covering a different but equally serious business called show business.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Mobile as the main driver for choosing a testing platform

    Many of the leading enterprises in the world are seeing mobile as the #1 growth vector in their app development efforts. as such, the key consideration for cloud-based testing (or any testing) should be "Mobile first", the amount of "new innovation" in non-mobile is declining significantly (when looking at 2012 and 2013 in terms of spent money).
    Mobile testing in enterprises, is still in its innovating phase, as a good practice, enterprises needs to understand the associated weight of each element (availability of devices, off-shoring, jailbreak, managed environment, ALM integration, security of devices, regulation (such as GLBA or HIPAA) compliance, etc) as they see it and less be influence by "innovation FUD". Testing is (and will always be) a "risk management" game and the risk can never be 0% unless you have unlimited resources at your hand.
  • Cloud testing tools must be able to keep up with the changing requirements

    Excellent article.

    The greatest benefit of cloud testing software is the ability to significantly reduce the time to test and simplify the testing process. This is what ultimately allows developers to quickly and efficiently deliver large quantities of solid software applications to meet the growing demand.

    Cloud testing tools must be able to keep up with the changing requirements of organizations, including the essential need for security and interoperability.

    As a cloud testing vendor (BlazeMeter), I can say that we invest a lot of thought into building tools that simplify and streamline the testing process taking into consideration the challenges enterprises face. Challenges such as testing behind firewall, the interoperability between cloud vendors or the integration with existing applications in the product life cycle.

    Looking forward to reading more,
    Alon Girmonsky
  • Nice Article

    Enjoyed the article, nice to see more of a balanced view on cloud verses installed testing tools. We see data security being a major issue for enterprise organizations with test cases being a window into how the application is architected and thus not something that they want in a cloud based environment. Cloud based testing for performance, stress or load though are interesting especially for globally accessible web applications / sites.
    I'm interested to see when or even if large enterprises will move into the cloud for good.
    Thanks again
    Bryce Day
    "Smart solutions for Quality Management"
  • Interesting article

    Hi Jamie

    The article is very interesting, please allow me to share my view on the mobile testing and its relation to cloud.

    As stated above the mobile world is dynamic and changes constantly (RIM is down, Microsoft Windows Phone is up, iOS6 is released, soon a new Android KLP might come out etc.).
    Keeping track of the fragmented world and supporting each OS version for dozens of phones while on each we may want to test on various browsers (If we test a web mobile application) makes things pretty much impossible by the R&D teams.
    The cloud service provides unique capabilities to the enterprises by allowing him to increase the test coverage platforms, perform test automation or manual testing in a shorter time, and with recorded and well documented test results.
    Saying that, we do want to assure that what we test and on what device we perform the test is accurate device with official ROM/OS version otherwise this will be indeed an issue.
    Last, the mobile cloud solutions (such as PerfectoMobile) does provide a secure based solutions for the enterprises either through VPN or other secured solutions, as well provides additional tools in the cloud such as "Device Clean Up" before exiting the cloud to assure that all date was cleaned and removed from the device under test