EMA (Enterprise Management Associates) recently released a report titled “Disrupting the Economics of Software Testing Through AI.” In this report, author Torsten Volk, Managing Research Director at EMA, discusses the reasons why traditional approaches to software quality cannot scale to meet the needs of modern software delivery. He highlights 5 key categories of AI and 6 critical pain points of test automation that AI addresses. 

We sat down with Torsten to talk about the report and hear his insights on how AI is impacting Software Testing:

1. What’s wrong with the current state of testing? 
   
   Many software development teams are struggling to deliver on the promise of Agile/DevOps and are faced with numerous technical challenges, such as rising application complexity and explosion of browsers/devices. Multiply this by the constant drive for faster releases to deliver increased customer value – without the ability to hire additional quality control staff – and you can quickly see why the traditional approach to software testing can no longer scale to keep up. We need to optimize the process with AI to eliminate the mundane and repetitive tasks and ultimately ensure business success.

2. How does AI help with software testing?
   
   There are five key capabilities AI provides to help improve human efficiency: smart test creation, self healing, coverage detection, anomaly detection, and visual inspection. In the report, I discuss the six critical pain points where these capabilities are delivering ROI today: false positives, test maintenance, inefficient feedback loops, rising application complexity, device sprawl, and tool chain complexity.
   
   

   The 5 Key AI Capabilities

   Of the capabilities available, AI-driven visual inspection has the broadest reach and highest impact. This discipline aims to provide test engineers with an additional “pair of eyes,” leaving the engineer to focus on activities that really need human intelligence. It provides humans with the contextual information needed to accelerate their test and remediation efforts, recommending solutions wherever necessary and remembering human decisions.

3. You talk about visual inspection having the highest impact. What pain points does it address?
   
   Traditional pixel-based comparison attempts to perform visual inspection but is plagued with false positives. Training deep learning models to inspect an application through the eyes of the end-user removes a lot of the mundane repetitive tasks that cause humans to be inefficient. Due to its accuracy, AI-driven visual inspection can enable teams to create a stable suite of automated tests with reduced false positives. Visual inspection can be further leveraged to drive cross-browser/cross-device validation at scale.

4. How are people adopting AI? Do I need to hire AI experts or develop an AI practice? 
   
   The world’s top brands are already adopting AI to transform their software testing, but they’re focusing on the ROI gained from the speed and scalability of leveraging AI in the development process – not developing the AI itself. There are solutions on the market today that give you the ability to take advantage of fully trained models and several can be layered on top of your existing test automation. One of the solutions that seems widely adopted for visual inspection is Applitools, which is already trained on +1B images and delivers extremely high levels of accuracy.

5. How does the role of the developer / tester change with the introduction of AI?
   
   Developers and testers still need to make a decision about what and how something should be automated. AI minimizes the mundane and repetitive tasks, freeing the engineers to do more creative, interesting and valuable work.
   
   For example, what happens when a tester is reviewing thousands of webpages for accuracy? Some pages have differences, but they don’t matter. AI can filter those out and highlight only the subset that needs review.
   
   Auto-classification, grouping issues together, is another great example of being able to reduce your work and improve accuracy. If I’m looking at what looks like the same problem over and over again, I might miss a defect. Whereas if the AI categorizes and groups similar issues together it greatly reduces the possibility of a missed defect.

6. What’s the future state for software testing, and where do we start?
   
   Autonomous testing is the vision for the future, but we have to ask ourselves, why don’t we have an autonomous car yet? It’s because today, we’re still chaining together models, and models of models. We’re striving to get to the point where AI is taking care of all of the tactical and repetitive decisions and humans are thinking more strategically at the end of the process, where they are more valuable from a business-focused perspective. I would recommend starting with the mature solutions available today in the areas of visual inspection and self healing – keeping an eye on the future and ongoing evolution of AI. 

Thanks to Torsten for spending the time with us. If you would like to start implementing AI as part of your testing pipeline, sign up for a free Applitools account and start enjoying the benefits of Visual AI in minutes.

This article was originally published on SD Times.

The post Disrupting the Economics of Software Testing Through AI: An Interview with Torsten Volk appeared first on Automated Visual Testing | Applitools.

From facial recognition to self-driving cars, Artificial Intelligence (AI) and machine learning (ML) have become commonplace for many industries in recent years. In parallel, the software development industry has undergone a transformation of its own.

As customers look to engage more through digital experiences, businesses have been forced to evolve faster than ever before. Enticing and delighting customers in every aspect of product delivery has become “business critical,” determining if the customer chooses, and continues, to do business with you over a competitor. Although the discipline of Quality Engineering has remained unchanged, every aspect of how quality is delivered has evolved. Businesses can no longer trade off quality vs. speed, as both quality and speed must be achieved for modern digital-first businesses.

Recently, two reports came out that speak directly to the intersection of these two trends and discuss how industry leaders are leveraging AI to modernize their approach to Quality Engineering in 2021 and beyond.

The first, from EMA (Enterprise Management Associates), is titled Disrupting the Economics of Software Testing Through AI. In this report, author Torsten Volk, Managing Research Director at EMA, discusses the reasons why traditional approaches to software quality cannot scale to meet the needs of modern software delivery. He highlights 5 key categories of AI and 6 critical pain points of test automation that AI addresses.

In addition, over the last couple of months Sogeti has been releasing sections of their State of AI applied to Quality Engineering 2021-22 report (with still more sections to come through February 2022). This comprehensive report is created in partnership with leading technology providers to provide a detailed examination of the current state of artificial intelligence across many use-cases in the field of Quality Engineering and centers around a key question — how can AI make quality validation smarter?

As the application of AI to testing continues to advance, it is important to understand its potential and how it can help improve the quality, velocity, and efficiency of Quality Engineering activities. Below, I’ll discuss some of the highlights from both of these reports and what they define as the future of Quality Engineering.

The Emergence of Modern Quality Engineering and the Need for AI

First, let’s talk about the reason why traditional approaches to software quality and test automation are no longer sufficient. The first section of the Sogeti report gives an overview of the business pressure to release faster and increasingly complex technical environments. As Torsten discusses in his report, modern software development teams are faced with many challenges that have driven up the complexity and cost of quality, such as the explosion of device/browser combinations and application complexity. Multiply this by the number of releases per month and you can quickly see that the traditional test automation tools can no longer scale to the challenges of modern software delivery.

The biggest problem with the traditional approach to test automation is that it scales linearly. The more, or faster, you need to test the more human and non-human resources you need — which only works if you have an infinite amount of resources (do you?).

With this in mind, the EMA and Sogeti reports discuss the ways modern organizations can leverage AI/ML to streamline their test automation practices and scale to meet the increased pace of software delivery.

AI Scales Test Automation to Identify Issues Before They Impact Users

When it comes to Quality Engineering, certain tools are capable of controlling the graphical user interface (GUI) or an application programming interface (API). Others analyze coverage and recommend additional actions, and some analyze log files in search of specific behaviors. These are just a few examples. But to increase developer productivity, there needs to be an understanding of each of these tasks and how they can be optimized.

How does this relate to AI? AI has the ability to apply algorithms and approaches used in tools to perform human-like tasks. For example, a developer can reason by examining an application to determine whether or not it has been properly tested. If the testing cycle has fallen short, they can then determine what additional testing needs to be done. AI has the potential to act in a similar manner. Although AI may require some training, once it has been trained it has the potential to continue to test the function even as the application evolves.

The EMA report details five key AI capabilities that can help organizations streamline and automate parts of their quality and testing workflow:

• Test Creation/Smart Crawling: Automatic discovery of new and changed test requirements through the continuous analysis of changes in the application and natural language process (NLP) of documented requirements
• Self-Healing: Continuous and automated remediation of broken test workflows
• Visual Inspection: Training of deep learning models to inspect the application through the eyes of the end user
• Coverage Detection: Automatic detection of the different paths that end users can take through the application and reporting of gaps in code coverage
• Anomaly Detection: Automatic detection of system behavior that is inconsistent with the predictions of the AI/ML model

The report highlights the key advantage of each capability and then details how the capabilities can bridge the gap between the “ideal scenario” and “in real life” situation for six critical pain points of Test Automation: false positives, test maintenance, insufficient feedback, application complexity, device/use-case coverage and toolchain complexity.

Each capability is assigned a rating, ranking their current impact in 2021 and predicting their future impact in 2024. Visual inspection, implemented with Visual AI, has the highest rating for both current and future impact with the key advantage that it “Provides complete and accurate coverage of the user experience. It learns and adapts to new situations without the need to write and maintain code-based rules.”

The EMA report goes on to add that “Smart crawling, self-healing, anomaly detection, and coverage detection each are point solutions that help organizations lower their risk of blind spots while decreasing human workload. Visual inspection (Visual AI) goes further compared to these point solutions by aiming to understand application workflows and business requirements.”

As discussed in the most recent section of the Sogeti report, Shorten Release Cycles with Visual AI, Visual AI is already a mature technology, currently being adopted by leading brands across industries to accelerate the delivery of their digital experiences. The high levels of accuracy, and ability to handle dynamic and shifting content, ensures teams do not get overwhelmed with false positives. The automated grouping and categorization of regressions, coupled with root cause analysis, accelerates feedback and reduces test maintenance efforts. Visual AI provides test engineers with an additional “pair of eyes,” leaving them to focus on areas that really need human intelligence — the power and impact of this approach is enormous.

From AI assistance to Autonomous Testing

Currently the industry is focused on having AI remove repetitive and mundane tasks, freeing humans to focus on the creative/complex tasks that require human intelligence. And as Torsten mentions in his report, “AI-based test automation technologies can deliver real ROI today and have the potential to address, and ultimately eliminate, today’s critical automation bottlenecks.”

The ROI will further increase as we look to the future and the next big innovation for Quality Engineering, Autonomous Testing. Autonomous Testing will change the role of developers and testers from testing the application to training the AI how to use the application, leaving it to perform the testing activities, and then reviewing the results. This change will deliver a fundamental increase in team efficiency, reducing the overall cost of quality and enabling businesses to establish truly scalable Quality Engineering practices.

Editor’s Note: This post first appeared on devopsdigest.com.

The post How AI is Making Test Automation Smarter appeared first on Automated Visual Testing | Applitools.

Here at Applitools, we are relentlessly focused on making software testing easier, faster and more reliable. Our industry-leading Visual AI, trained on more than a billion images to deliver 99.9999% accuracy, can automatically detect visual and functional bugs just as a human would. The blazing fast Ultrafast Test Cloud, powered by our Ultrafast Grid, makes cross-browser, cross-device and cross-platform testing a snap. And we’re not done there, as we continue to grow and innovate along the path to autonomous testing.

Don’t just take our word for it though.  Hundreds of clients have determined that Applitools is the best visual testing tool and are using Applitools today, including nine of the top 10 software companies in the world, seven of the top 10 banks in the US and two of the top three retailers. We’ve empirically measured the impact of Visual AI and we know that the results – 5.8x faster test creation, 5.9x more efficient code (in terms of lines of code), 3.8x improvement in test code stability and 45% increase in effectiveness catching bugs early – speak for themselves.

We know that Applitools can make a dramatic difference in the lives of our customers – but again, you don’t need to just hear this from us. Numerous organizations have recognized the value Applitools brings just this month.

Update 1/19/22: We’re excited to share that we have also just been named the Best Testing Service/Tool in the DevOps Dozen² Awards for 2021. We’re honored by this latest recognition as well as the ones below!

Applitools Awarded Leading Vendor in Testing for North America

Software Testing News recently named Applitools Leading Vendor. This award is granted only to the vendor who receives top marks for both their product/service and customer service. Software Testing News specifically looked for commitment to a high quality product with excellent customer satisfaction, strong value for its cost, rock-solid reliability and ease of use, and proof of thought leadership that can drive the software testing/QE industry forward. 

It’s not easy to score highly on each of those metrics, and we’re thrilled that Software Testing News has recognized us with this award as the as the leading software testing vendor.

Applitools Named Fastest-Growing Software Testing Vendor in North America

It’s one thing to be recognized for product innovation and customer service – and make no mistake, we are proud of that – but when the rubber meets the road it is only the businesses that are growing that can meet the increasingly complex needs of more customers over the long term. That’s why we’re so excited to be named the fastest-growing software testing vendor in North America by Deloitte. 

This placement on the Deloitte Technology Fast 500 is a testament to the demand in the market for a solution that can make the demanding lives of test engineers easier and significantly increase the reliability of software.

Applitools a Leader on GetApp and Capterra

We’re always grateful to the testing community and our customers for their feedback, and we’re pleased that many of them are not shy about sharing their love for Applitools and how it helps make their lives easier and more productive. That’s why we’re a Category Leader in Automated Testing on GetApp and have a strong 4.6/5.0 rating on Capterra – with no reviewer giving us fewer than four stars.

Our commitment to our customers is ironclad. We know that it’s only by our continued dedication to giving our customers exactly what they need that we can continue to win awards for growth and innovation, and we’re honored that our customers choose to share their experiences with the world.

Try it for Yourself

See for yourself why everyone from our customers to market-defining publications are calling Applitools a leader in automated software testing and showcasing the tremendous impact we’re having on the software testing industry. Get your free account and dive in, or reach out to schedule your demo today.

The post Applitools Recognized as Testing Leader by DevOps.com, Deloitte, GetApp and More appeared first on Automated Visual Testing | Applitools.

We are proud to announce our position as the top ranked software testing vendor on the Deloitte Technology Fast 500, a ranking of the 500 fastest-growing technology, media, telecommunications, life sciences, fintech, and energy tech companies in North America.

Here at Applitools we know that when people experience the power of Visual AI, they fall in love, and that there is a large and growing demand for this technology in the market. We’re proud to share our recognition as the fastest-growing software testing company on the continent as a part of the Deloitte Fast 500. This long-running list, now in its 27^th year, provides a ranking of the fastest-growing technology, media, telecommunications, life sciences, fintech, and energy tech companies — both public and private — in North America. Technology Fast 500 award winners are selected based on percentage fiscal year revenue growth from 2017 to 2020.

It is an honor to be the top ranked software testing vendor on the 2021 Deloitte Technology Fast 500. And, it is a testament to the growing demand for Visual AI technology. From healthcare to transportation, digital experiences are becoming central to companies’ success. Applitools makes delivering flawless digital experiences simple, ensuring brand integrity is never compromised. Thanks to all of the hard work and dedication from the Applitools team that made our ranking on the Technology Fast 500 possible.

Gil Sever, CEO and co-founder, Applitools

How Does Applitools Help?

Modern software development teams are rapidly delivering innovation to market through more frequent and shorter release cycles, but they struggle to fully test the customer experience due to increasing application complexity and an explosion of device/browser combinations.

Applitools is helping over 400 of the world’s top digital brands release, test, and monitor flawless mobile, web, and native apps in a fully automated way. Our Visual AI mimics the human eye and brain to avoid undetected functional and visual bugs, minimizing false positive bug alerts. The Ultrafast Test Cloud can instantly validate entire application pages and detect issues on even the most complex and dynamic pages. When development teams are confident they can fix functional and visual bugs faster, they can push more high-quality code faster than ever before.

Applitools’ Visual AI modernizes critical test automation use cases — functional and visual regression testing, web and mobile UI/UX testing, cross browser / cross device testing, localization testing, PDF testing, digital accessibility and legal/compliance testing — to transform the way businesses deliver innovation at the speed of DevOps without jeopardizing their brand.

An Honor to be Recognized

The Technology Fast 500 list is an impressive list of top companies in the tech space, and we’re honored to be recognized and proud of our placement as one of the fastest growing companies in North America.

Each year the Technology Fast 500 shines a light on leading innovators in technology and this year is no exception. In the face of innumerable challenges resulting from the pandemic, the best and brightest were able to pivot, reinvent and transform and grow. We celebrate the winning organizations and especially the talented employees driving their success.

Paul Silverglate, vice chair, Deloitte LLP and U.S. technology sector leader

“The pandemic has underscored the urgent need for tech solutions in a variety of areas across health care, fintech, energy tech, entertainment, to name a few, so reliance on innovators like the winners of the Technology Fast 500 is more important than ever,” said . “These companies are not only at the cutting edge, transforming the way we do business, but most importantly, recognize the strategic importance of ongoing innovation, especially in the ever-changing world of technology.” 

Christie Simons, partner, Deloitte & Touche LLP and industry leader for technology, media and telecommunications within Deloitte’s audit & assurance practice

Thank You

Many dedicated team members have worked long and hard here at Applitools to achieve this ranking, but it is also the direct result of the feedback and collaboration we’ve had with our customers – we could not have done it without you. Thank you for trusting us to deliver flawless automated testing for you, and we’re excited to head into the future of testing together.

Learn more

For more, you can read our full press release, or schedule a demo and see for yourself how Visual AI is helping industry leaders delivery visually perfect digital experiences.

The post Applitools Named Fastest-Growing Software Testing Vendor in North America appeared first on Automated Visual Testing | Applitools.

We are excited to announce the latest release of Applitools Eyes, 10.13. A big focus of this release is helping teams work efficiently, collaborate, and receive notifications on visual changes via the communication systems they are using everyday.

Along with the new and expanded integration options, we’ve added some additional improvements that we hope you’ll find useful!

Using Microsoft Teams to collaborate with your colleagues? You can now get Eyes test results directly to your Microsoft Teams chat

In addition to sharing test results on both Slack and via email, the new Applitools Eyes-Microsoft Teams integration provides you with the option to receive and view your test results via your Microsoft Teams chat. The Applitools Eyes App sends notifications to your Microsoft Teams chat to inform you when batches have finished running and to share a results summary with you. Learn more.

Enhanced baseline creation for new browsers and OS versions 

New browser or OS versions often introduce visual differences, therefore it is important to test across multiple versions to ensure visual perfection across all screens. Applitools Eyes now supports efficient and simple testing of your application on new browsers and OS versions. To save you time and effort, Eyes identifies the most relevant baseline to reuse whenever you test on a new version. It also allows you to easily filter and group baselines according to the browser or OS version you would like to explore. This capability is enabled by default for new accounts, for existing users please contact support to turn on this new & important capability for your accounts. Learn more.

And there is more…

• Receive more focused test results notifications via Slack integration – Slack notifications can now be sent according to selected batch properties.
• Ensure perfect UI with each commit with GitHub integration – update the test status on all commits.
• Enhanced Rally integration – in addition to opening issues from within the Eyes dashboard, now automatically trigger an update in Rally when closing issues in Eyes.
• Benefit from UX improvements for marking multiple regions – keep the draw mode active when adding multiple regions of the same type.

Explore this new release to find out more! Existing customers can upgrade today for free, or if you’re new to Applitools feel free to explore the latest features with a free trial below.

The post Announcing Applitools Eyes 10.13: Enhanced Team Collaboration, Baseline across Browser/OS Versions and More appeared first on Automated Visual Testing | Applitools.