Disrupting the Economics of Software Testing Through AI: An Interview with Torsten Volk

EMA (Enterprise Management Associates) recently released a report titled “Disrupting the Economics of Software Testing Through AI.” In this report, author Torsten Volk, Managing Research Director at EMA, discusses the reasons why traditional approaches to software quality cannot scale to meet the needs of modern software delivery. He highlights 5 key categories of AI and 6 critical pain points of test automation that AI addresses. 

We sat down with Torsten to talk about the report and hear his insights on how AI is impacting Software Testing:

1. What’s wrong with the current state of testing? 
   
   Many software development teams are struggling to deliver on the promise of Agile/DevOps and are faced with numerous technical challenges, such as rising application complexity and explosion of browsers/devices. Multiply this by the constant drive for faster releases to deliver increased customer value – without the ability to hire additional quality control staff – and you can quickly see why the traditional approach to software testing can no longer scale to keep up. We need to optimize the process with AI to eliminate the mundane and repetitive tasks and ultimately ensure business success.

2. How does AI help with software testing?
   
   There are five key capabilities AI provides to help improve human efficiency: smart test creation, self healing, coverage detection, anomaly detection, and visual inspection. In the report, I discuss the six critical pain points where these capabilities are delivering ROI today: false positives, test maintenance, inefficient feedback loops, rising application complexity, device sprawl, and tool chain complexity.
   
   

   The 5 Key AI Capabilities

   Of the capabilities available, AI-driven visual inspection has the broadest reach and highest impact. This discipline aims to provide test engineers with an additional “pair of eyes,” leaving the engineer to focus on activities that really need human intelligence. It provides humans with the contextual information needed to accelerate their test and remediation efforts, recommending solutions wherever necessary and remembering human decisions.

3. You talk about visual inspection having the highest impact. What pain points does it address?
   
   Traditional pixel-based comparison attempts to perform visual inspection but is plagued with false positives. Training deep learning models to inspect an application through the eyes of the end-user removes a lot of the mundane repetitive tasks that cause humans to be inefficient. Due to its accuracy, AI-driven visual inspection can enable teams to create a stable suite of automated tests with reduced false positives. Visual inspection can be further leveraged to drive cross-browser/cross-device validation at scale.

4. How are people adopting AI? Do I need to hire AI experts or develop an AI practice? 
   
   The world’s top brands are already adopting AI to transform their software testing, but they’re focusing on the ROI gained from the speed and scalability of leveraging AI in the development process – not developing the AI itself. There are solutions on the market today that give you the ability to take advantage of fully trained models and several can be layered on top of your existing test automation. One of the solutions that seems widely adopted for visual inspection is Applitools, which is already trained on +1B images and delivers extremely high levels of accuracy.

5. How does the role of the developer / tester change with the introduction of AI?
   
   Developers and testers still need to make a decision about what and how something should be automated. AI minimizes the mundane and repetitive tasks, freeing the engineers to do more creative, interesting and valuable work.
   
   For example, what happens when a tester is reviewing thousands of webpages for accuracy? Some pages have differences, but they don’t matter. AI can filter those out and highlight only the subset that needs review.
   
   Auto-classification, grouping issues together, is another great example of being able to reduce your work and improve accuracy. If I’m looking at what looks like the same problem over and over again, I might miss a defect. Whereas if the AI categorizes and groups similar issues together it greatly reduces the possibility of a missed defect.

6. What’s the future state for software testing, and where do we start?
   
   Autonomous testing is the vision for the future, but we have to ask ourselves, why don’t we have an autonomous car yet? It’s because today, we’re still chaining together models, and models of models. We’re striving to get to the point where AI is taking care of all of the tactical and repetitive decisions and humans are thinking more strategically at the end of the process, where they are more valuable from a business-focused perspective. I would recommend starting with the mature solutions available today in the areas of visual inspection and self healing – keeping an eye on the future and ongoing evolution of AI. 

Thanks to Torsten for spending the time with us. If you would like to start implementing AI as part of your testing pipeline, sign up for a free Applitools account and start enjoying the benefits of Visual AI in minutes.

This article was originally published on SD Times.