Editor’s note: This article was written with the support of ChatGPT.
Last week, Applitools hosted Unlocking the Power of ChatGPT and AI in Test Automation: Next Steps, when I discussed how artificial intelligence – specifically ChatGPT – can impact the field of test automation. The webinar delved into the various applications of AI in test automation, the benefits it brings, and the best practices to follow for successful implementation. With the ever-growing need for efficient and effective testing, the webinar is a must-watch for anyone looking to stay ahead of the curve in software testing. This blog article recaps the key takeaways from the webinar. Also, you can find the full recording, session materials, and more in our event archive.
Takeaways from the previous webinar
I started with a recap of the takeaways from the previous webinar, Unlocking the Power of ChatGPT and AI in Testing: A Real-World Look. The webinar focused on two main aspects from a testing perspective: testing approach mindset (strategy, design, automation, and execution) and automation perspective. ChatGPT was able to help with automation by guiding me to automate test cases more quickly and effectively. ChatGPT was also able to provide solutions to programming problems, such as giving a solution to a problem statement and refactoring code. However, there were limitations to ChatGPT’s ability to provide answers, particularly in terms of test execution, and some challenges when working with large blobs of code.
If you didn’t catch the previous webinar, you can still watch it on demand.
What’s new in AI since the previous webinar
Since we hosted the previous webinar, there have been many updates in the AI chatbot space. A few key updates we covered in the webinar include:
- ChatGPT has become accessible on laptops, phones, and Raspberry Pi, and can be run on own devices.
- Google Bard was released, but it is limited to English language, cannot continue conversations, and cannot help with coding.
- ChatGPT 4 was released, which accepts images and text inputs and provides text outputs.
- ChatGPT Plus was introduced, offering better reasoning, faster responses, and higher availability to users.
- Plugins can now be built on top of ChatGPT, opening up new and powerful ways of interaction.
During the webinar, I gave a live demo of some of ChatGPT’s updates, where it was able to provide code implementation and generate unit tests for a programming question.
Using AI to address common challenges in test automation
Next, I discussed the actual challenges in automation and how we can leverage AI tools to get better results. Those challenges include:
- Slow and flaky test execution
- Sub-optimal, inefficient automation
- Incorrect, non-contextual test data
Using AI to address flakiness in test automation
The section specifically focused on flaky tests related to UI or locator changes, which can be identified using consistent logging and reporting. I advised against using a retry listener to handle flaky tests and instead suggest identifying and fixing the root cause. I then demonstrated an example of a test failing due to a locator change and discussed ways to solve this challenge.
Read our step-by-step tutorial to learn how to use visual AI locators to target anything you need to test in your application and how it can help you create tests that are more resilient and robust.
Using AI to address sub-optimal or inefficient information
Next, I discussed sub-optimal or inefficient automation and how to improve it. I use GitHub Copilot to generate a new test and auto-generate code. I explained how to integrate GitHub Copilot and JetBrains Aqua with IntelliJ and how to use Aqua to find locators for web elements. Then, I showed how to implement code in the IDE and interact with the application to perform automation.
Using AI to address incorrect, non-contextual test data
Next, I discussed the importance of test data in automation testing and related challenges. There are many libraries available for generating test data that can work in the context of the application. Aqua can generate test data by right-clicking and selecting the type of text to generate. Copilot can generate data for “send keys” commands automatically. It’s important to have a good test data strategy to avoid limitations and increase the value of automation testing.
Potential pitfalls of AI
AI is not a magic solution and requires conscious and contextual use. Over-reliance on AI can lead to incomplete knowledge and lack of understanding of generated code or tests. AI may replace certain jobs, but individuals can leverage AI tools to improve their work and make it an asset instead of a liability.
Data privacy is also a major concern with AI use, as accidental leaks of proprietary information can occur. And AI decision-making can be problematic if it does not make the user think critically and understand the reasoning behind the decisions. Countries and organizations are starting to ban the use of AI tools like ChatGPT due to concerns over data privacy and accidental leaks.
Conclusion
Overall at first, I was skeptical about AI in automation, but that skepticism has reduced significantly. You must embrace technology or you risk being left behind. Avoid manual repetition, and leverage automation tools to make work faster and more interesting. The automation cocktail (using different tools in combination) is the way forward.
Focus on ROI and value generation, and make wise choices when building, buying, or reusing tools. Being agile is important, not just following a methodology or procedure. Learning, evolving, iterating, communicating, and collaborating are key to staying agile. Upskilling and being creative and innovative are important for individuals and teams. Completing the same amount of work in a shorter time leads to learning, creativity, and innovation.
Be sure to read my next article where I answers questions from the audience Q&A. If you have any other questions, be sure to reach out on Twitter or LinkedIn.
