With great speed comes great responsibility: Software testing now a continuous race

Survey finds enterprises refresh software at least once day, but less than one-third support continuous testing.
Written by Joe McKendrick, Contributing Writer

Continuous integration and continuous delivery is giving us software updates every day in many cases. A recent survey of 500 IT executives finds 58 percent of enterprises deploy a new build daily, and 26 percent at least hourly. That's why Agile and DevOps are so important. With great speed comes great responsibility. A constant stream of software needs constant quality assurance. To make sure things are functioning as they should, organizations are turning to continuous testing. 

Photo:Joe McKendrick

That's the takeaway from the survey, released by Capgemini and Sogeti, in collaboration with Broadcom. Continuous testing, the process of fast and efficient validation of software releases in agile developments through highly automated tests, is gaining ground in large enterprises, with close to one-third of IT executives (32 percent) stating that their IT departments had "fully embraced continuous testing."  

Also: What is DevOps? An executive guide to agile development and IT operations 

While it may seem obvious that continuous testing requires a great deal of automation, the study also found that automation was only being used to execute 24 percent of test cases, 24 percent of end-to-end business scenarios, and to generate 25 percent of the required test data. More than one-third (36 percent) said that most of their testing time is currently spent searching, managing, maintaining and generating test data. . 

Artificial intelligence to the rescue? AI can enable smart test orchestration. While it was not clear how deeply AI was employed in current continuous testing cycles, at least 28 percent of executives say the intend to introduce AI to manage the process. "With the addition of machine learning capabilities, systems will be able to automatically determine the tests that are required in the release and production cycles," the survey's authors explain. 

Finally, another important piece of the continuous testing challenge is keeping the process as transparent as possible.Among the executives surveyed, 35% identified a "complete audit trail of testing activities" and a "consolidated test and release pipeline" as the most important test orchestration capabilities. Another 32 percent pointed to a need for a "single-place for cross-team collaboration" and "continuous delivery pipeline visibility."

The Capgemini authors make the following recommendations to enable successful continuous testing:

  • Creative visibility of quality processes by implementing customized QA dashboards across CI/CD pipelines.
  • Set up automated testing cycles, to detect defects early in the lifecycle.
  • Leverage AI and machine learning to optimize test cycles. 
  • Automate the self-provisioning of test data.
Editorial standards