We’ve partnered with SelectorsHub to bring more automated visual testing tips to your daily testing feed. In this article, we’ll talk about some tools to help you test efficiently and stay on top of testing trends and techniques.

TestingDaily is a web extension that serves as a hub for software testers and quality assurance professionals. It provides a wide range of resources, including news, articles, interviews, and tools related to software testing and QA.

TestingDaily aims to provide a comprehensive platform for testers to learn, connect, and stay up-to-date with the latest trends and best practices in the field. TestingDaily also features a job board where testers can search for new career opportunities in the industry. Overall, it’s a valuable resource for anyone looking to improve their testing skills and stay informed about the latest developments in software testing.

Test Automation University (TAU) offers community-curated test automation resources and free courses to improve the skills of testers everywhere. Powered by Applitools, TAU is an online learning platform that offers dozens of courses on testing, automation, and programming from some of the world’s best instructors. Whether you take a single course, follow one of the pre-defined learning paths, or build your own curriculum, you will grow your skills and boost your career.

Not sure where to start? Here are 7 Awesome Test Automation University Courses Not To Miss.

SelectorsHub offers a web scraping tool that helps users generate accurate and efficient CSS, XPath, and regex selectors for web elements. It is an open-source, browser extension that can be used with Chrome, Firefox, and Edge.

With SelectorsHub, users can quickly create selectors without needing to inspect the source code, allowing them to speed up their web scraping process. The tool also provides features such as selector validation, highlighting, and grouping to make the selector creation process more convenient and efficient. SelectorsHub can be particularly helpful for those who work with web data extraction, web automation, or web testing.

Learn how the SelectorsHub tool can increase efficiency without compromising learning skills in this on-demand session with creator Sanjay Kumar.

Finally, Applitools Automation Cookbook offers free bite-sized video tutorials on common test automation tasks in Cypress, Playwright, and Selenium. Created by automation experts Angie Jones and Colby Fayock, each of the more than 25 “recipes” include video demonstrations, sample code, and transcripts. You can learn how to execute tests in headless mode, work with iFrames, sort and verify tables, and much more.

Last week, Applitools hosted Unlocking the Power of ChatGPT and AI in Test Automation: Next Steps, where I explored how artificial intelligence, specifically ChatGPT, can revolutionize the field of test automation. In this article, I’ll share the audience Q&A as well as some of the results of the audience polls. Be sure to read my previous article, where I summarized the key takeaways from the webinar. You can also find the full recording, session materials, and more in our event archive.

Audience Q&A

Our audience asked various questions about the data and intellectual property when using AI and adding AI into their own test automation processes.

Intellectual properties when using ChatGPT

Question: To avoid disclosing company intellectual properties to ChatGPT, is it not better to build a “private” ChatGPT / large language model to use and train for test automation inside the company while securing the privacy?

My response: The way a lot of organizations are proceeding is setting up private ChatGPT-like infrastructure to get the value of AI. I think that’s a good way to proceed, at least for now.

Data privacy when using ChatGPT

Question: What do you think about feeding commercial data (requirements, code, etc.) to ChatGPT and/or OpenAI API (e.g. gpt-3.5-turbo) data privacy connected to the recent privacy issues like with Samsung, exposure of ChatGPT chats, and so forth?

My response: Feeding public data is okay, because it’s out in the public space anyway, but commercial data could be public or it could be private and that could become an issue. The problem is we do not understand enough about how ChatGPT is using the data or the questions that we are asking it. It is constantly learning, so if you feed a very unique type of question that it has never come across before, the algorithm is intelligent to learn from that. It might give you the wrong answer, but it is going to learn based on your follow-up questions, and it is going to use that information to answer someone else’s similar question.

Complying with data regulations

Question: How can we ensure that AI-driven test automation tools maintain compliance with data privacy regulations like GDPR and CCPA during the testing process?

My response: It’s a tough question. I don’t know how we can ensure that, but if you are going to use any AI tool, you must make sure you are asking very focused, specific questions that don’t disclose any confidential information. For example, in my demo, I had a piece of code pointing to a website asking it a very specific question how to implement it. That question could be implemented using some complex free algorithms or anything else, but taking that solution, you make it your own and then implement it in your organization. That might be safer than disclosing anything more. This is a very new area right now. It’s better to be on the side of caution.

Adding AI into the test automation process

Question: Any suggestions on how to embed ChatGPT/AI into automation testing efforts as a process more than individual benefit?

My response: I unfortunately do not have an answer to this yet. It is something that needs to be explored and figured out. One thing I will add is that even though it may be similar to many others, each product is different. The processes and tech stacks are going to vary for all these types of products you use for testing and automation, so one solution is not going to fit everyone. Auto-generated code will go up to a certain level, but at least as of now, the human mind is still very essential to use it correctly. So it’s not going to solve your problems; it is just going to make solving them easier. The examples I showed are ways to make it easier in your own case.

Using AI for API and NFRs

Question: How effective would AI be for API and NFR?

My response: I could ask a question to give me a performance test implementation approach for the Amazon website, and it gives me a performance test strategy. If I ask it a question to give me an implementation detail of what tool I should use, it is probably going to suggest a few tools. If I ask it to build an initial script for automating this performance test, it is probably going to do that for me as well. It all depends on the questions you are asking to proceed from there, and I’m sure you’ll get good insights for NFRs.

Using AI a dedicated private cloud instance

Question: Our organization is very particular about intellectual property protection, and they might deny us from using third-party cloud tools. What solution is there for this?

My response: Applitools uses the public cloud. I use a public cloud for my learning and training and demos that I do, but a lot of our customers actually use the dedicated cloud instance, which is hosted only for them. Only they have access to that, so that takes care of the security concerns that might be there. We also work with our customers to ensure from compliance and security perspectives that all the questions are answered and to make sure everything conforms as per their standards.

Using AI for mobile test automation

Question: Do you think AI can improve quality of life for automation engineers working on mobile app testing too? Or mostly web and API?

My response: Yes, it works for mobile. It works for anything that you want. You just have to try it out and be specific with your questions. What I learned from using ChatGPT is that you need to learn the art of asking the questions. It is very important in any communication, but now it is becoming very important in communicating with tools as well to get you the appropriate responses.

Audience poll results

In the live webinar, the audience was asked “Given the privacy concerns, how comfortable are you using AI tools for automation?” Of 105 votes, over half of the respondents would be somewhat or very comfortable with using AI tools for automation.

• Very comfortable: 16.95%
• Somewhat comfortable: 33.05%
• Not sure: 24.59%
• Somewhat comfortable: 14.41%

Next steps

You can take advantage of AI today by using Applitools to test web apps, mobile apps, desktop apps, PDFs, screenshots, and more. Applitools offers SDKs that support several popular testing frameworks in multiple languages, and these SDKs install directly into your projects for seamless integration. You can try it yourself by claiming a free account or request a demo.

Check out our events page to register for an upcoming session with our CTO and the inventor of visual AI Adam Carmi. For our clients and friends in the Asia-Pacific region, be sure to register to attend our upcoming encore presentation of Unlocking the Power of ChatGPT and AI in Test Automation happening on April 26th.

Editor’s note: This article was written with the support of ChatGPT.

Last week, Applitools hosted Unlocking the Power of ChatGPT and AI in Test Automation: Next Steps, when I discussed how artificial intelligence – specifically ChatGPT – can impact the field of test automation. The webinar delved into the various applications of AI in test automation, the benefits it brings, and the best practices to follow for successful implementation. With the ever-growing need for efficient and effective testing, the webinar is a must-watch for anyone looking to stay ahead of the curve in software testing. This blog article recaps the key takeaways from the webinar. Also, you can find the full recording, session materials, and more in our event archive.

Takeaways from the previous webinar

I started with a recap of the takeaways from the previous webinar, Unlocking the Power of ChatGPT and AI in Testing: A Real-World Look. The webinar focused on two main aspects from a testing perspective: testing approach mindset (strategy, design, automation, and execution) and automation perspective. ChatGPT was able to help with automation by guiding me to automate test cases more quickly and effectively. ChatGPT was also able to provide solutions to programming problems, such as giving a solution to a problem statement and refactoring code. However, there were limitations to ChatGPT’s ability to provide answers, particularly in terms of test execution, and some challenges when working with large blobs of code.
If you didn’t catch the previous webinar, you can still watch it on demand.

What’s new in AI since the previous webinar

Since we hosted the previous webinar, there have been many updates in the AI chatbot space. A few key updates we covered in the webinar include:

• ChatGPT has become accessible on laptops, phones, and Raspberry Pi, and can be run on own devices.
• Google Bard was released, but it is limited to English language, cannot continue conversations, and cannot help with coding.
• ChatGPT 4 was released, which accepts images and text inputs and provides text outputs.
• ChatGPT Plus was introduced, offering better reasoning, faster responses, and higher availability to users.
• Plugins can now be built on top of ChatGPT, opening up new and powerful ways of interaction.

During the webinar, I gave a live demo of some of ChatGPT’s updates, where it was able to provide code implementation and generate unit tests for a programming question.

Using AI to address common challenges in test automation

Next, I discussed the actual challenges in automation and how we can leverage AI tools to get better results. Those challenges include:

• Slow and flaky test execution
• Sub-optimal, inefficient automation
• Incorrect, non-contextual test data

Using AI to address flakiness in test automation

The section specifically focused on flaky tests related to UI or locator changes, which can be identified using consistent logging and reporting. I advised against using a retry listener to handle flaky tests and instead suggest identifying and fixing the root cause. I then demonstrated an example of a test failing due to a locator change and discussed ways to solve this challenge.

Read our step-by-step tutorial to learn how to use visual AI locators to target anything you need to test in your application and how it can help you create tests that are more resilient and robust.

Using AI to address sub-optimal or inefficient information

Next, I discussed sub-optimal or inefficient automation and how to improve it. I use GitHub Copilot to generate a new test and auto-generate code. I explained how to integrate GitHub Copilot and JetBrains Aqua with IntelliJ and how to use Aqua to find locators for web elements. Then, I showed how to implement code in the IDE and interact with the application to perform automation.

Using AI to address incorrect, non-contextual test data

Next, I discussed the importance of test data in automation testing and related challenges. There are many libraries available for generating test data that can work in the context of the application. Aqua can generate test data by right-clicking and selecting the type of text to generate. Copilot can generate data for “send keys” commands automatically. It’s important to have a good test data strategy to avoid limitations and increase the value of automation testing.

Potential pitfalls of AI

AI is not a magic solution and requires conscious and contextual use. Over-reliance on AI can lead to incomplete knowledge and lack of understanding of generated code or tests. AI may replace certain jobs, but individuals can leverage AI tools to improve their work and make it an asset instead of a liability.

Data privacy is also a major concern with AI use, as accidental leaks of proprietary information can occur. And AI decision-making can be problematic if it does not make the user think critically and understand the reasoning behind the decisions. Countries and organizations are starting to ban the use of AI tools like ChatGPT due to concerns over data privacy and accidental leaks.

Conclusion

Overall at first, I was skeptical about AI in automation, but that skepticism has reduced significantly. You must embrace technology or you risk being left behind. Avoid manual repetition, and leverage automation tools to make work faster and more interesting. The automation cocktail (using different tools in combination) is the way forward.

Focus on ROI and value generation, and make wise choices when building, buying, or reusing tools. Being agile is important, not just following a methodology or procedure. Learning, evolving, iterating, communicating, and collaborating are key to staying agile. Upskilling and being creative and innovative are important for individuals and teams. Completing the same amount of work in a shorter time leads to learning, creativity, and innovation.

Be sure to read my next article where I answers questions from the audience Q&A. If you have any other questions, be sure to reach out on Twitter or LinkedIn.

My answers to questions I received at TAU Conference 2023

Recently it was a pleasure to give a talk about the role of automation in Mobile Continuous Testing at the TAU Conference powered by Applitools, and actually, it was an excellent opportunity to meet different test automation experts and attendees from all over the world.

If you missed the session, you could find the on-demand recording here: applitools.info/kte

During the session, I got various questions about mobile testing and CI, so I wrote this blog to answer them all.

Let’s get started!

Question 1: Automating tests for different devices

Question: What is the most efficient approach when automating tests for the same mobile app on different (Android/iOS) devices? Automate with a tool that works for both or a tool for each one individually?

My answer: The best answer always depends on the team’s needs and goals. Using a tool that works for both platforms is a good choice, because this approach saves time and effort in test automation as it allows you to write test scripts once and run them on multiple devices and platforms such as using Appium. 

But sometimes, the team decides to use platform-specific testing tools such as Espresso for Android and XCUITest for iOS, because they need the mobile developers to work closely with the test engineers in writing the UI tests because they believe it’s a shared responsibility. 

Additionally, they can use the same programming language. Test scripts will be in the same repositories, and using the native locators from the app will be directly accessible in this case.

Question 2: Testing multi-platform apps

Question: Is the approach to/process for testing a multi-platform app like Evernote (mobile, desktop, web) different from a mobile-only app?

My answer: Testing a multi-platform app like Evernote requires a different approach because the following reasons:

• Each platform’s user interface, operating system, hardware, and software requirements (mobile, desktop, web) differ, so testing must account for these differences and ensure that the app operates as intended.
• To ensure consistency across all platforms, including features like note-taking, search, sync, and collaboration, each platform must be tested separately, as well as its integration.
• Testing the app across various platforms ensures the design, navigation, and features are consistent and intuitive.

In order to know how to deal with different platforms, these things should be included in the test plan and strategy at the beginning.

Question 3: Upcoming course content

Question: Is the outline described in your presentation today part of an existing course? Or will it be part of a future course re-haul of your existing course on TAU?
My answer: No, it’s not in my existing courses, but maybe we can have it in the future.

Question 4: Manual testing for mobile CI

Question: When in this mobile CI workflow would manual testing fit in? Or is it suggested that manual testing is not needed?

My answer: If your team would like to achieve the fully automated CI/CD to build, test, package, and release the mobile apps to the App Stores, manual testing will not fit in the lifecycle because the goal is to make the process fully automated as much as we can. But if the target is to have continuous integration only, you can build and package the apps and then send them to the manual testers to do the testing activities. After that, if there are no issues in the build or the release, you can continue with the release process of manually automating it. 

It always depends on the team goal and the release cadence. Some teams or companies release every one or two weeks; in this case, manual testing will be a blocker and releases can take a long time.

Question 5: Manual testing with an automation setup

Question: Is there still a need for manual testing, even with an automation setup?

My answer: Yes, manual testing can be used to identify issues not covered by automated tests, sometimes, different scenarios require a human eye, interaction, or decision. 

It always depends on the type of mobile app and the functionalities we need to cover; this can be added to the test plan and the test strategy from the beginning.

Question 6: Making apps testable

Question: Isn’t it also necessary that apps are made testable?

My answer: Yes, I totally agree, as a mobile development team, we should consider making the mobile app testable by doing the following tips:

• By writing modular code, you can test individual components independently.
• Make your app’s logic, data, and presentation layers separate using design patterns like MVVM or MVP. This will make testing easier.
• Decouple dependencies between classes through dependency injection. This will make it easier to swap out components during testing.

Question 7: Using tools for mobile test automation

Question: If you’re not strong in mobile app automation, for something like a React Native app, would you stick with something like Appium (which is familiar with native automation/Selenium web), or would it be better to try the Maestro (or something similar)?

My answer: The POC (Proof of Concept) project is always the best, in my opinion. If you already have a React Native app, you can try to create a project and try to automate the app with Appium, as you know, Appium is a black-box testing tool so at the end you need only the .apk or .ipa files to be able to automate them, which is not different with React Native apps because at the end you will have already two native apps (iOS and Android). Or you can try Maestro. It’s a new framework, and it may be beneficial in your case.

Question 8: Staying up-to-date on industry skills

Question: Does the coming of flutter make the tester’s life easy?

My answer: I cannot guarantee that, as a tester, we face new challenges every day. For instance, we must learn about new testing tools and technologies, which requires us to stay on top of our game. In contrast, as we can see these days, AI is booming and is affecting our daily work, which makes acquiring unique skills indispensable.

Question 9: Preparing for ISTQB Foundation Certification

Question: How best to prepare for an ISTQB Foundation Certification?

My answer: It depends on which ISTQB certificate you want to achieve, for instance, if it’s the foundation level, you need at least 6 months of experience with software testing to understand the fundamentals and the concepts in the Syllabus. 

As you can see in the following image, the certificates have different tracks and levels. Each level required specific requirements and a different level of experience. You can find all the details on the ISTQB official website, including the Syllabus and the sample questions with answers.

Besides reading the Syllabus, your background will be enough.

Wrapping up

I hope you enjoyed these answers to your questions, and I look forward to seeing you at the TAU and Applitools conferences in the near future.

Please do not hesitate to contact me if you have any other questions.

My LinkedIn: https://www.linkedin.com/in/moataz-nabil/

My Twitter: @moatazeldebsy

Thank you for reading.

Happy testing!

That’s right, Test Automation University (TAU) is holding a conference! It’ll be a two-day virtual event from March 8–9, 2023 hosted by Applitools. We will sharpen our software quality skills as we go all-in for the testing community. This article gives you nine reasons why you should attend this awesome event.

#1: Find out what’s new with TAU

TAU is alive and kicking, and we’ve got some incredible new developments in the works. In my opening keynote address, I’ll dish out all the details for upcoming courses, community engagements, and learning paths. There might even be a few surprises!

#2: Sharpen your skills with tutorials

Our first day will be stacked with six two-and-a-half-hour tutorials so you can really hone your automation skills. Want to stack up Cypress and Playwright? Filip Hric and I have you covered. Need to drop down to the API layer? Julia Pottinger’s got the session for you. Unit testing? Tariq King will help you think inside the box. Wanna crank it up with load and performance testing? Leandro will show you how to use k6. How about some modern Selenium with visual testing? Anand Bagmar’s our man. It’s the perfect time to sharpen your skills.

#3: Hear inspiring talks from our instructors

There’s no doubt that our TAU instructors are among the best in software testing. We invited many of them to share their knowledge and their wisdom on our second day. Our program is designed to inspire you to step up in your careers.

#4: Test your testing skills

Want to challenge your testing skills together with your other TAU classmates? Well, Carlos Kidman will be offering a live “testing test,” where everyone gets to submit answers in real time. The student with the highest score will win a pretty cool prize, so be sure to study up!

#5: Algorave at the halftime show

Have you ever listened to algorithmically-generated music that is coded live in front of you? It’s an experience! Dan Gorelick will kick out the algorave jams during our halftime show. (Glow sticks not included.)

#6: Cram even more content with lightning talks

Lightning talks bring bite-sized ideas to the front stage in rapid succession. They’re a welcome change of pace from full-length talks. We’ll have a full set of lightning talks from a few of our instructors along with a joint Q&A where you can ask them anything!

#7: Compete in extracurricular games

Every university needs some good extracurricular activities, right? We’ll have a mid-day game where you, as members of the audience, will get to participate! We’re keeping details under wraps for now so that nobody gets an unfair advantage.

#8: Win awards like a homecoming champion

What would a celebration be without superlative awards? We will recognize some of our best instructors and students in our closing ceremony. You might also win some prizes based on your participation in the conference!

#9: Get your conference tickets for FREE!

Applitools is hosting the 2023 TAU Conference as a free community event for everyone to attend. Just like our TAU courses, you don’t need to pay a dime to participate. You don’t even need to be a TAU student! Come join us for the biggest block party in the testing community this year. Register here to reserve your spot.

The AI promise

AI is not a new technology. Recently, the field has made huge advancements.

Currently it seems like AI technology is mostly about using ML (machine learning) algorithms to train models with large volumes of data and use these trained computational models to make predictions or generate some outcomes.

From a testing and test automation point-of-view, the question in my mind still remains: Will AI be able to automatically generate and update test cases? Can it find contextual defects in the product? Can it eventually inspect code and the test coverage to prevent defects getting into production?

ICYMI: Read my recent article on this topic, AI: The magical helping hand in testing.

The promising future

Recently I came across a lot of interesting buzz created by ChatGPT, and I had to try it out. 

Given I am a curious tester by heart, I signed up for it on https://chat.openai.com/ and tried to see the kind of responses generated by ChatGPT for a specific use case.

Live demo: Watch me demonstrate how to use ChatGPT for testing and development in the on-demand recording of Unlocking the Power of ChatGPT and AI in Testing: A Real-World Look.

What is the role of AI in testing?

I started with a simple question to ChatGPT. The answer was interesting. But the answer was nothing different than what I already knew.

So I thought of asking some more specific questions.

Create a test strategy using AI

I asked ChatGPT to create a test strategy for testing and automating Amazon India shopping app and website.

I was blown away by the first response. Though I can’t use this directly, as it is quite generic, the answer was actually pretty good as a starting point.

I was now hooked, and I had to keep going on now.

What test cases should I automate for my product?

Example: What test cases should I automate for Amazon India?

Like the test strategy, these are not to the level of details I am looking for. But it’s a great start.

That said, at the top of the test automation pyramid, I do not want to automate test cases, but I want to automate test scenarios. So, I asked ChatGPT about the important scenarios I should automate.

What test scenarios should I automate for my product?

Though not the specific test scenarios I was expecting, there is a clear difference between the identified test cases and the test scenarios.

Code generation: the first auto-generated test

I asked ChatGPT to give me the Selenium-Java automation code to automate the “search and add a One Plus phone to cart scenario” for Amazon India website.

ChatGPT not only corrected my question, but also gave me the exact code to implement the above scenario.

I updated the question to generate the test using Applitools Visual AI, and voila, here was the solution:

Is this code usable?

Not really!

If you run the test, there is a good chance it would fail. The reason being, the generated code assumes the page is “rendered” as soon as the actions are done. To make this code usable and consistent in execution, we need to update it with realistic and intelligent waits at relevant places to cater to the rendering and loading time required by the browser, and of-course, your environment.

NOTE: Intelligent waits are important as opposed to a hard wait to optimise the execution time. Also, being realistic in your wait durations is very important. You do not want to wait for a particular state of the application for more than what is expected to happen in production.

Also, typically in our test frameworks, we have various abstraction layers and utilities to manage different functionalities and responsibilities. But we can use snippets of this generated code and easily plug it in the right places in our frameworks. So it is still very valuable.

I also tried the same for some internal sites – i.e. not available to the general public. The test was still created, but not sure about the validity of those tests.

Test data generation

Lastly, I asked specific questions around test data – an aspect unfortunately ignored or forgotten until the last minute by most teams.

Example: What test data do I need to automate test scenarios for my product?

Again, I was blown away by the detailed test data characteristics that were given as a response. It was specific about the relevant data required for product, user, payment, and order to automate the important test scenarios that were highlighted in the earlier response. 

I tried to trick ChatGPT to “search and add a product to cart” – but I didn’t specify what product.

It caught my bluff and gave very clear instructions that I should provide valid test data.

Try it yourself: Can ChatGPT accurately translate your tests from one framework to another? Try it out and let us know how it went @Applitools.

Other examples

Generating content

Instead of thinking and typing this blog post, I could have used ChatGPT to generate the post for me as well.

I didn’t even have time to step away to get a cup of coffee while this was being generated!

Planning a vacation

Another interesting use case of ChatGPT – getting help to plan a vacation!

NOTE: Be sure to work with your organization’s security team and @OpenAI before implementing ChatGPT into the organization’s official processes.

What’s next? What does it mean for using AI in testing?

In my examples, the IDEs are very powerful and make it very easy for programmers and automation engineers to write code in a better and more efficient way.

36.8% of our audience participants answered that they were worried about AI replacing their jobs. Others see AI as a tool to improve their skills and efficiency in their job.

Tools and technologies like ChatGPT will offer more assistance to such roles. Individuals can focus on logic and use cases. I have doubts on the applicability of these tools to provide answers based on contextual information. However, given very focused and precise questions, tools can provide information to help implement the same in an efficient and optimal fashion.

I am hooked!

To learn more, watch on-demand recording of Unlocking the Power of ChatGPT and AI in Testing: A Real-World Look. Be looking for a follow-on event in early April where I will go deeper into specific use cases and how ChatGPT and its use in testing are evolving. Sign up for an email notification when registration opens.

Learn how to future-proof your test automation pipeline with Cypress and Applitools by adding tests that run from GitHub Actions. In this article, we’ll share how to ensure your test automation pipeline can scale while staying reliable and easy to maintain.

Automating different types of tests

To illustrate our different types of test automation, we’ll be using the example project Cypress Heroes. In this full-stack TypeScript app, users can take the following actions:

• Log in with an email and password
• Like heroes, which increments the hero’s number of fans
• Hire heroes, which increments the hero’s number of saves
• Manage hero profile information like name, superpowers, and price

ICYMI: Watch the on-demand recording of Future-Proofing Your Automation Pipeline to see Ely Lucas from Cypress demo the example project.

End-to-end testing

Cypress is traditionally known for end-to-end testing. You automate user interactions for specific scenarios from start to finish in the browser, and then run functional assertions to check the state of elements at each step. End-to-end tests are hidden in an actual web server and hit the site just like a user would.

Measurable stats for code coverage of your end-to-end testing can act as a health metric for your website or app. Adding coverage reports to your automation pipeline as commits can help ensure you’re testing all parts of your code.

Component testing

If you’re using a component-based framework like React or Angular or a design system like Storybook, you can also do component testing to test UI components. In this example, we have a button component with a few tests that pass, the hero card test, and a test for the login form. These components are being mounted in isolation outside of your typical web server.

Think of component tests as “UI unit” tests. While they don’t give end-to-end coverage, they’re quick and easy to run.

API testing

For your back end, you’ll need to automate API tests. The example project is using a community-built plugin called cypress-plugin-api. This plugin provides an interface inside the Cypress app to test APIs. It’s really cool and it’s super fun, and it allows you to write tests that you would have to do manually in a tool like Postman.

Fun fact: Cypress Ambassador Filip Hric developed the cypress-plugin-api. Check out Filip’s Test Automation University courses.

The API tests in our example are in the separate server project. We can use the command npx cypress open, and then we can run those tests in Chrome. We can see all of our results that we’re getting the response of the status codes. We can view a post request, the headers that were sent, the headers that were returned, and other stuff that you normally get from a tool like Postman.

And it’s just baked into the app, which is really nice. Cypress is basically a web app that tests a web app. And so, you could extend Cypress as an app with things like this to help you do your testing and to have it all seamlessly integrated.

Running a pipeline with GitHub Actions

The example project uses GitHub Actions to set up the test automation pipeline. When working on smaller projects, it’s easy to have CI interactions baked into your repository, all in one place.

Configuring your GitHub Action

With GitHub Actions, you declare everything you need in a YAML file in the .github/workflows folder. Your actions become part of your repository and are covered by version control. If you make any changes, you can review them easily with a simple line-by-line diff. GitHub Actions make it easy to automate processes alongside other interactions you make with your repository. For example, if you open a pull request, you can have it automatically kick off your tests and do linting. You can even perform static code analysis before merging changes.

Some environment variables are set at the top of the YAML file. The API URL is what the client app uses to communicate to the API. The example app is hooked up to send test results to the Cypress Cloud. Those results can then be used for analytics, diagnostics, reporting, and optimizing our test workflows. The Cypress cloud also requires a GitHub token, so it can do things like correctly identify what pull request is being merged.

For those new to GitHub Actions: You can define environment variables per step in a job, but declaring them at the top helps you update them painlessly.

Running each of our tests

To keep things simple, there is only one job right now in this GitHub Action. First, it checks out the code straight from GitHub. Next, it builds the project using the Cypress GitHub Action. The Cypress GitHub Action does a few things for you like building your application or npm installing or yarn installing the dependencies.

Building first means that subsequent jobs don’t have to build the app again. We’ve set run test to false, which is a parameter to the Cypress GitHub Action, because we don’t want to run the tests here. We’ll be running the tests separately below.

We have our component tests in our GitHub Action. We tell it to install false, since we installed it up above. And then we run our custom test command, which opens Cypress in run mode and initiates component testing. This record tells the GitHub Action to send the results to Cypress Cloud.

And then we have to start the client and server. For both end-to-end tests and API tests, the application must be up and running component tests. For the end-to-end test and API tests, the example app is hitting live servers.

This run command will start both the React app and the Node server, and then it will run the end-to-end test. We’re telling it again to not install the dependency, since it was already installed. Then, we’re running the command to start the end-to-end testing. The wait command will wait to make sure that both the client URL and the API URL are both up and running before it will start the test. If the test starts before both URLs get up and running, you’ll have some tests fail.

Another thing that the Cypress GitHub Action does is that you have the option to wait for these services to be live before the testing starts. By default, the npm run test commands are going to use the Chromium browser built into Electron. If you want to test on other browsers, you must make sure those browsers are installed on the runner. Cypress provides Docker images that you can add to your configuration to download the different browsers. However, downloading additional browsers increases the file size and makes the runs take longer.

Make sure that the Cypress binary itself is downloaded and installed. It’s going to run headlessly. This is because the command set up in these scripts is run mode, which is headless, whereas open mode is with the UI.

And then it will run the API test, which is very similar to end-to-end tests, except that since we’re not hitting the actual client app – only hitting the API app – we’re only waiting to make sure that the API URL is up and running.

If you write test cases per the local database for end-to-end testing before pushing to GitHub Actions, someone else running those test cases on their system could potentially fail. In whatever kind of test automation you develop, you’ll need to handle test data properly to avoid collisions. There are many different strategies you can follow. For more information on this and solving sample data dependency, watch my talk Managing the Test Data Nightmare.

How long do the test suites take?

When running your tests with Cypress and GitHub Actions, the results are uploaded to Cypress Cloud. You can go into Cypress Cloud and actually watch replays of all these tests that happened. The entire pipeline run in the example was 3 minutes and 50 seconds for all three test suites.

The individual test suites we ran took the following times:

• Component tests: 49 seconds
• End-to-end tests: 1 minute and 18 seconds
• API tests: 13 seconds

Improving test coverage with visual assertions

Since all the Cypress tests are run inside of the browser window, you can visually see them and inspect to make sure that they’re looking correctly. But this type of review is a manual step. If someone accidentally makes a change to the stylesheet, the site could no longer be running properly, but if we run the tests, they’ll pass.

We can use Applitools Eyes to fix this issue.

Visual testing is meant to automate the things that traditional automation is not so good at. For example, as long as particular IDs on your page are in the DOM somewhere, your traditional automation scripts with something like Cypress are still going to find and interact with the elements. Applitools Eyes uses visual AI to look at an app and be able to detect these kinds of visual differences that traditional assertions struggle to capture. Let’s add some visual snapshots to these end-to-end tests.

Adding Applitools to your project

First, you’ll need an Applitools account. You can register a free Applitools account with your GitHub username or your email, and you’ll be good to go. You’ll need your Applitools API key for when we run tests with Applitools.

Next, we’ll need to install the Applitools SDK using npm install @applitools/eyes-cypress.

It can be a dev dependency or it can be a regular dependency – whichever you prefer. In the example project, we use a dev dependency. In the example project, we’re using the Applitools Eyes SDK for Cypress, we have Applitools SDKs for basically every tool framework you got.
Next, we’ll need to create an Applitools configuration file. Where in Cypress projects, you have your cypress.config.js file, basically we want one that’s called applitools.config.js.

Configuring your Applitools runner

In the Applitools config file, we will specify the configuration for running visual tests. There’s a separation between declaring configuration and actually adding test steps.
One of the settings we want is called batchName, and we’re going to set that to “cy heroes visual tests” to reflect the name of our demo app. The batch name will appear in the Eyes Test Manager (or the Applitools “dashboard”) after we run our visual tests.

Next, we’ll set the browsers. This will be a list, with each item being an entry that specifies a browser configuration, including name, width, and height.

Typically, since Cypress runs inside of an Electron app, it can be challenging to test mobile browsers. However, the Applitools Ultrafast Grid enables us to render our visual snapshots on mobile devices. The settings for mobile devices are going to be a bit different than those for browsers. Instead of having a name, we’re going to have a device name.

Our applitools.config.js file is complete. When we run our tests – either locally or in the GitHub Action – Applitools will render the snapshots it captures on these four browser configurations in the Ultrafast Grid and report results using the batch name. Furthermore, the local platform doesn’t matter. Even if you run this test on Windows, the Ultrafast Grid can still render snapshots on Safari and mobile emulators. A snapshot is just going to be a capture of that full page. Applitools will do the re-rendering with the appropriate size, the appropriate browser configuration, and all that will happen in the cloud. Essentially you can do multi-browser and multi-platform testing with simple declarations.

Now that we have completed the configuration, let’s update the tests to capture visual snapshots, starting with the homepage.

Setting up our test suites

You need to make sure that your tests aren’t interfering with other tests. In these tests, we’re going through and modifying some of the heroes that are in the application. The state of the application changes per test, so to get around that, we’re creating a new hero just for working with our tests and deleting the hero after the tests.

In the example, we’re using Cypress tasks, which is code that actually runs on the Node process part of Cypress. It’s directly communicating with our database to add the hero, delete the hero, and all the other types of setup tasks that we want to do before we actually run our test.

So it’s going to happen for each of the tests, and then we’re visiting the homepage and getting access to the hero.

We get our new hero and then we call cy.deleteHero, which is going to call the database to delete the hero. From the describe block at the start of every test, we get our hero. And then, finally, we have the hero card by its name, and we find the button that has the right selectors, so we can actually select it and click the button.

This test is making sure that you’re logged in before you can like this hero. We’re making sure that the modal popped up, clicking the okay on the modal, and then making sure that modal disappears and does not exist anymore.

Down below we have another suite for when a normal user is logged in. And so we’re using a custom Cypress command to log in with this username and password. You can define these custom commands that are like making your own function, encapsulating a little bit of logic so that it could be reusable.

So what we’re doing to test the login is going to the homepage, running the login process, and verifying the login was actually successful. The cy.session is caching a session for us to restore the session later from cookies. This helps speed up your test so you’re not having to go through the whole flow of actually logging in again.

We have another suite here for when an admin user is logged in, because an admin user can edit users and delete heroes.

 In the example, negative login tests – where you use the wrong username and/or password – are under the component tests.

In the login form component test, when an email and password is invalid, an error should show. The example uses cy.intercept to mock the API request that goes to the cert, which goes to the off endpoint and returns a status code 401, which represents an invalid login.

You can either write a component test or an end-to-end test. In this case, a component test makes it easier to set up the mock data.

Adding a call to Applitools Eyes

With the test suites set up, we’re ready to add some visual snapshots here. We need to call an Eyes session using the Applitools Eyes SDK. The idea is that we open our eyes, and we can take visual snapshots. And then at the end of the test, we will close our eyes to say that we’ve captured all the snapshots for that session or for that test. And at that point, Applitools Eyes will upload the snapshots that are captured to the Applitools Eyes server, do all of the re-rendering of the things of those four browsers in the Ultrafast Grid. Then we can log into the Applitools dashboard and we can see exactly what happened with our testing.
To get the autocomplete for Eyes commands, we need to set up the Applitools Eyes stuff with the Cypress project. We already did npm install on the package, so we’ll need to run npx eyes-setup.

We’ll want to use the command cy.eyesOpen in the homepage describe block under the beforeEach method. We want to pass an app name and test name for logging and reporting purposes. You might also put their Cypress eyes open code in the beforeEach of the test cases, so the call doesn’t need to be duplicated.

Then, in the afterEach block, you’ll call cy.eyesClose.

In this test, you must log in, make sure that the modal pops up, log in, and then click okay in the modal and make sure the modal disappears, so we’ll need a snapshot when the modal is up and one when the modal goes away. In this case, we’ll capture the whole window.

If we didn’t want to capture everything, we could actually capture a region, like a div or even an individual element. On a small scale, using the region option does not make a measurable difference in execution speed, but it gives you a way to tune the type of snapshot we want.

For capturing the next step, we can basically copy the whole call there and paste it, changing the tag to homepage with the modal dismissed.

These snapshots are very straightforward to write, and something that we could consider is that some of those other assertions you might arguably be able to remove. The visual snapshot is going to capture everything on that window, so if it’s there and visible, we’re going to capture it and track it over time.

You would still need to keep all of your interactions, but you can remove most of your assertions checking visible elements. However, there are certainly things where if you want to check a very specific numeric value, you still want to keep those assertions.

Running the updated tests

All we need to do to run this test is make sure that we have our Applitools API key from our account saved as an environment variable of the Cypress application.

Note: If you happen to steal someone’s API key, it doesn’t really help you. It just means they’ll see your results, and you won’t. API keys should be kept secret and safe.

Using the Applitools Eyes dashboard

So to see the visual testing results, we will need to view them in the Applitools Eyes dashboard.

You can view your test results in a grid to see the UI quickly, or you can view your results in a list to see your configurations quickly.

On the left, you’ve got the batch name that was set. Then on the main part of the body, you’ll see there are actually four tests. We only wrote one test, but each test is run once per browser configuration we specified, providing cross-browser and cross-platform testing without additional steps.

If we open up the snapshots, you can see the two snapshots that we captured. These results are new, because this is the first time we’ve run the test.

We’ve established the snapshot as a baseline image, meaning anything in the future will be checked against that.

That’s where that visual aspect of the testing comes in. Your Cypress results will essentially tell you if it was bare bones basic functional, and then Eyes will tell you what it actually looked like. You get richer results together.

Resolving test results in the dashboard

Let’s see what this looks like if we make that visual change.

In the main file, we’ve updated the stylesheet and run the test again. There is no need to do anything in the Applitools Eyes dashboard before re-running the test.

The new test batch is in an unresolved state because Eyes detected a visual difference. In theory, a visual difference could be good or bad. You could be making an intentional change. Visual AI is basically a change detector that makes it obvious to you, the human, to decide what is good or bad. Then anytime Applitools Eyes sees the same kind of passing or failing behavior in the future, it’ll remember.

It’s important to note that the unresolved test results won’t stop your test automation job or your automation flow. Test automation would complete normally. You as the human tester would review visual test results in the Eyes Test Manager (the “dashboard”) afterwards. The pipeline would not wait for you to manually mark visual test results.

Let’s open up one of those snapshots so we can see it full screen.

In the upper left, below the View menu in the ribbon, there’s a dropdown to show both so that you can see the baseline and test side by side.

In the example, we had removed the stylesheet, so we can see very clearly that it’s very different. It’s not always this obvious. In this case, pretty much the whole screen is different. But if it were like a single button that was missing or something shunted a little bit, it would show that a specific area was different. That’s the power of the visual AI check.

Whenever Applitools detects a visual change, you can mark it as “passing” with a thumbs-up. Then that snapshot automatically becomes the new baseline against which future checkpoints are compared. Applitools will go to the background and track similar images. And it will automatically update those appropriately as well.

Note: If you ever want to “reset” snapshots, you can also delete the baselines and run your tests “fresh” as if for the first time. The snapshots they capture will automatically become new baseline images.

Once we’ve resolved all test results, we’ll need to save. And now if we were to rerun our test again, Applitools Eyes would see the new snapshots and pass tests as appropriate. If you have dynamic content or test data, you add region annotations, which will ignore anything in the region box.
It is possible to compare your production and staging environments. You can use our GitHub Integration to manage different branches or versions of your application. We also support different baselines for A/B testing.

Closing thoughts

That’s basically how you would do visual testing with Applitools and Cypress. There are two big points to remember if you want to add visual testing to your own test suites:

• To get these tests running in your pipeline, the only change you’d have to make is to inject the Applitools API key in those environment variables.
• We didn’t really add a fourth suite of tests. Visual testing is more of a technique or an aspect of testing, not necessarily its own category of tests. All you have to do is work in the SDK, capture some snapshots, and you’re good to roll.

We hope this guide has helped you to build out your test automation pipeline to be more reliable and scalable. If you liked the guide, check out our Applitools tutorials for other guides on building your test automation pipeline. Watch the on-demand recording of Future-Proofing Your Automation Pipeline to see the full walkthrough. To keep up-to-date with test automation, you can peruse our latest courses taught by industry-leading testing experts on Test Automation University. Happy testing!

Let the Engineers Speak! Part 5: Audience Q&A

Articles — 01.11.2023

Let the Engineers Speak! Part 4: Changing Frameworks

Articles — 01.05.2023

Let the Engineers Speak! Part 3: Favorite Test Integrations

Articles — 12.28.2022

Right before the end of 2022, Cypress surprised us with their new major release: version 12. There wasn’t too much talk around it, but in terms of developer experience (DX), it’s arguably one of their best releases of the year. It removes some of the biggest friction points, adds new features, and provides better stability for your tests. Let’s break down the most significant ones and talk about why they matter.

No more “detached from DOM” errors

If you are a daily Cypress user, chances are you have seen an error that said something like, “the element was detached from DOM”. This is often caused by the fact that the element you tried to select was re-rendered, disappeared, or detached some other way. With modern web applications, this is something that happens quite often. Cypress could deal with this reasonably well, but the API was not intuitive enough. In fact, I listed this as one of the most common mistakes in my talk earlier this year.

Let’s consider the example from my talk. In a test, we want to do the following:

1. Open the search box.
2. Type “abc” into the search box.
3. Verify that the first result is an item with the text “abc”.

As we type into the search box, an HTTP request is sent with every keystroke. Every response from that HTTP request then triggers re-rendering of the results.

The test will look like this:

it('Searching for item with the text "abc"', () => {
 
 cy.visit('/')
 
 cy.realPress(['Meta', 'k'])
 
 cy.get('[data-cy=search-input]')
   .type('abc')
 
 cy.get('[data-cy=result-item]')
   .first()
   .should('contain.text', 'abc')
 
})

The main problem here is that we ignore the HTTP requests that re-render our results. Depending on the moment when we call cy.get() and cy.first() commands, we get different results. As the server responds with search results (different with each keystroke), our DOM is getting re-rendered, making our “abc” item shift from second position to first. This means that our cy.should() command might make an assertion on a different element than we expect.

Typically, we rely on Cypress’ built-in retry-ability to do the trick. The only problem is that the cy.should() command will retry itself and the previous command, but it will not climb up the command chain to the cy.get() command.

It is fairly easy to solve this problem in versions v11 and before, but the newest Cypress update has brought much more clarity to the whole flow. Instead of the cy.should() command retrying only itself and the previous command, it will retry the whole chain, including our cy.get() command from the example.

In order to keep retry-ability sensible, Cypress team has split commands into three categories:

• assertions
• actions
• queries

These categories are reflected in Cypress documentation. The fundamental principle brought by version 12 is that a chain of queries is retried as a whole, instead of just the last and penultimate command. This is best demonstrated by an example comparing versions:

// Cypress v11:
cy.get('[data-cy=result-item]') //  not retried
 .first() // retried
 .should('contain.text', 'abc') // retried
 
// Cypress v12:
cy.get('[data-cy=result-item]') //  retried
 .first() // retried
 .should('contain.text', 'abc') // retried

cy.get() and cy.first() are commands that both fall into queries category, which means that they are going to get retried when cy.should() does not pass immediately. As always, Cypress is going to keep on retrying until the assertion passes or until a time limit runs up.

cy.session() and cy.origin() are out of beta

One of the biggest criticisms of Cypress.io has been the limited ability to visit multiple domains during a test. This is a huge blocker for many test automation engineers, especially if you need to use a third-party domain to authenticate into your application.

Cypress has advised to use programmatic login and to generally avoid trying to test applications you are not in control of. While these are good advice, it is much harder to execute them in real life, especially when you are in a hurry to get a good testing coverage. It is much easier (and more intuitive) to navigate your app like a real user and automate a flow similar to their behavior.

This is why it seems so odd that it took so long for Cypress to implement the ability to navigate through multiple domains. The reason for this is actually rooted in how Cypress is designed. Instead of calling browser actions the same way as tools like Playwright and Selenium do, Cypress inserts the test script right inside the browser and automates actions from within. There are two iframes, one for the script and one for the application under test. Because of this design, browser security rules limit how these iframes interact and navigate. Laying grounds for solving these limitations were actually present in earlier Cypress releases and have finally landed in full with version 12 release. If you want to read more about this, you should check out Cypress’ official blog on this topic – it’s an excellent read.

There are still some specifics on how to navigate to a third party domain in Cypress, best shown by an example:

it('Google SSO login', () => {
 
 cy.visit('/login') // primary app login page
 
 cy.getDataCy('google-button')
   .click() // clicking the button will redirect to another domain
 
 cy.origin('https://accounts.google.com', () => {
   cy.get('[type="email"]')
     .type(Cypress.env('email')) // google email
   cy.get('[type="button"]')
     .click()
   cy.get('[type="password"]')
     .type(Cypress.env('password')) // google password
   cy.get('[type="button"]')
     .click()
 })
 
 cy.location('pathname')
   .should('eq', '/success') // check that we have successfully
 
})

As you see, all the actions that belong to another domain are wrapped in the callback of cy.origin() command. This separates actions that happen on the third party domain.

The Cypress team actually developed this feature alongside another one that came out from beta, cy.session(). This command makes authenticating in your end-to-end tests much more effective. Instead of logging in before every test, you can log in just once, cache that login, and re-use it across all your specs. I recently wrote a walkthrough of this command on my blog and showed how you can use it instead of a classic page object.

This command is especially useful for the use case from the previous code example. Third-party login services usually have security measures in place that prevent bots or automated scripts from trying to login too often. If you attempt to login too many times, you might get hit with CAPTCHA or some other rate-limiting feature. This is definitely a risk when running tens or hundreds of tests.

it('Google SSO login', () => {
 
 cy.visit('/login') // primary app login page
 cy.getDataCy('google-button')
   .click() // clicking the button will redirect to another domain
 
 cy.session('google login', () => {
   cy.origin('https://accounts.google.com', () => {
     cy.get('[type="email"]')
       .type(Cypress.env('email')) // google email
     cy.get('[type="button"]')
       .click()
     cy.get('[type="password"]')
       .type(Cypress.env('password')) // google password
     cy.get('[type="button"]')
       .click()
   })
 })
 
 cy.location('pathname')
   .should('eq', '/success') // check that we have successfully
 
})

When running a test, Cypress will make a decision when it reaches the cy.session() command:

• Is there a session called google login anywhere in the test suite?
  • If not, run the commands inside the callback and cache the cookies, local storage, and other browser data.
  • If yes, restore the cache assigned to a session called “google login.”

You can create multiple of these sessions and test your application using different accounts. This is useful if you want to test different account privileges or just see how the application behaves when seen by different accounts. Instead of going through the login sequence through UI or trying to log in programmatically, you can quickly restore the session and reuse it across all your tests.

This also means that you will reduce your login attempts to a minimum and prevent getting rate-limited on your third party login service.

Run all specs in GUI

Cypress GUI is a great companion for writing and debugging your tests. With the version 10 release, it has dropped support for the “Run all specs” button in the GUI. The community was not very happy about this change, so Cypress decided to bring it back.

The reason why it was removed in the first place is that it could bring some unexpected results. Simply put, this functionality would merge all your tests into one single file. This can get tricky especially if you use before(), beforeEach(), after() and afterEach() hooks in your tests. These would often get ordered and stacked in unexpected order. Take following example:

// file #1
describe('group 1', () => {
 it('test A', () => {
   // ...
 })
})
 
it('test B', () => {
 // ...
})
 
// file #2
before( () => {
 // ...
})
 
it('test C', () => {
 // ...
})

If this runs as a single file, the order of actions would go like this:

• before() hook
• test B
• test C
• test A

This is mainly caused by how Mocha framework executes blocks of code. If you properly wrap every test into describe() blocks, you would get much less surprises, but that’s not always what people do.

On the other hand, running all specs can be really useful when developing an application. I use this feature to get immediate feedback on changes I make in my code when I work on my cypress plugin for testing API. Whenever I make a change, all my tests re-run and I can see all the bugs that I’ve introduced. ?

Running all specs is now behind an experimental flag, so you need to set experimentalRunAllSpecs to true in your cypress.config.js configuration file.

Test isolation

It is always a good idea to keep your tests isolated. If your tests depend on one another, it may create a domino effect. First test will make all the subsequent tests fail as well. Things get even more hairy when you bring parallelisation into the equation.

You could say that Cypress is an opinionated testing framework, but my personal take on this is that this is a good opinion to have. The way Cypress enforces test isolation with this update is simple. In between every test, Cypress will navigate from your application to a blank page. So in addition to all the cleaning up Cypress did before (clearing cookies, local storage), it will now make sure to “restart” the tested application as well.

In practice the test execution would look something like this:

it('test A', () => {
 cy.visit('https://staging.myapp.com')
 // ...
 // your test doing stuff
})
 
// navigates to about:blank
 
it('test B', () => {
 cy.get('#myElement') // nope, will fail, we are at about:blank
})

This behavior is configurable, so if you need some time to adjust to this change, you can set testIsolation to false in your configuration.

Removing of deprecated commands and APIs

Some of the APIs and commands reached end of life with the latest Cypress release. For example, cy.route() and cy.server() have been replaced by the much more powerful cy.intercept() command that was introduced back in version 6.

The more impactful change was the deprecation of Cypress.Cookies.default() and Cypress.Cookies.preserveOnce() APIs that were used for handling the behavior of clearing up and preserving cookies. With the introduction of cy.session(), these APIs didn’t fit well into the system. The migration from these commands to cy.session() might not seem as straightforward, but it is quite simple when you look at it.

For example, instead of using Cypress.Cookies.preserveOnce() function to prevent deletion of certain cookies you can use cy.session() like this:

beforeEach(() => {
 cy.session('importantCookies', () => {
   cy.setCookie('authentication', 'top_secret');
 })
});
 
it('test A', () => {
 cy.visit('/');
});
 
it('test B', () => {
 cy.visit('/');
});

Also, instead of using Cypress.Cookies.defaults() to set up default cookies for your tests, you can go to your cypress/support/e2e.js support file and set up a global beforeEach() hook that will do the same as shown in the previous example.

Besides these there were a couple of bug fixes and smaller tweaks which can all be viewed in Cypress changelog. Overall, I think that the v12 release of Cypress is one of the unsung heroes. Rewriting of query commands and availability of cy.session() and cy.origin() commands may not seem like a big deal on paper, but it will make the experience much smoother than it was before.

New command queries might require some rewriting in your tests. But I would advise you to upgrade as soon as possible, as this update will bring much more stability to your tests. I’d also advise to rethink your test suite and integrate cy.session() to your tests as it might not only handle your login actions more elegantly but shave off minutes of your test run.

If you want to learn more about Cypress, you can come visit my blog, subscribe to my YouTube channel, or connect with me on Twitter or LinkedIn.

Let the Engineers Speak! Part 5: Audience Q&A

Articles — 01.11.2023

Let the Engineers Speak! Part 4: Changing Frameworks

Articles — 01.05.2023

Let the Engineers Speak! Part 3: Favorite Test Integrations

Articles — 12.28.2022

Check out the latest test automation videos from Applitools.

We hope you got to take time to rest, unplug, and spend time with your loved ones to finish out 2022 with gratitude. I have been incredibly appreciative of the learning opportunities and personal growth that 2022 offered. In reflection of our past quarter here at Applitools, we’ve curated our latest videos from some amazing speakers. If you missed any videos while away on holiday or finishing off tasks for the year, we’ve gathered the highlights for you in one spot.

ICYMI: Back in November, Andrew Knight (a.k.a. the Automation Panda) shared the top ten Test Automation University courses.

Cypress vs. Playwright: The Rematch

One of our most popular series is Let the Code Speak, where we compare testing frameworks in real examples. In our rematch of Let the Code Speak: Cypress vs. Playwright, Andrew Knight and Filip Hric dive deeper to how Cypress and Playwright work in practical projects. Quality Engineer Beth Marshall moderates this battle of testing frameworks while Andy and Filip explore comparisons of their respective testing frameworks in the areas of developer experience, finding selectors, reporting, and more.

Automating Testing in a Component Library

Visual testing components allows teams to find bugs earlier, across a variety of browsers and viewports, by testing reused components in isolation. Software Engineering Manager David Lindley and Senior Software Engineer Ben Hudson joined us last year to detail how Vodafone introduced Applitools into its workflow to automate visual component testing. They also share the challenges and improvements they saw when automating their component testing.

When to Shift Left, Move to Centre, and Go Right in Testing

Quality isn’t limited to the end of the development process, so testing should be kept in mind long before your app is built. Quality Advocate Millan Kaul offers actionable strategies and answers to questions about how to approach testing during different development phases and when you should or shouldn’t automate. Millan also shares real examples of how to do performance and security testing.

You, Me, and Accessibility: Empathy and Human-Centered Design Thinking

Inclusive design makes it easier for your customers with your varying needs and devices are able to use your product. Accessibility Advocate and Crema Test Engineer Erin Hess talks about the principles of accessible design, how empathy empowers teams and end users, and how to make accessibility more approachable to teams that are newer to it. This webinar is helpful all team members, whether you’re a designer, developer, tester, product owner, or customer advocate.

Erin also shared a recap along with the audience poll results in a follow-up blog post.

Future of Testing October 2022

Our October Future of Testing event was full of experts from SenseIT, Studylog, Meta, This Dot, EVERSANA, EVERFI, LAB Group, and our own speakers from Applitools. We covered test automation topics across ROI measurement, accessibility, testing at scale, and more. Andrew Knight, Director of Test Automation University, concludes the event with eight testing convictions inspired by Ukiyo-e Japanese woodblock prints. Check out the full Future of Testing October 2022 event library for all of the sessions.

Skills and Strategies for New Test Managers

Being a good Test Manager is about more than just choosing the right tools for your team. EasyJet Test Manager Laveena Ramchandani shares what she has learned in her experience on how to succeed in QA leadership. Some of Laveena’s strategies include how to create a culture that values feedback and communication. This webinar is great for anyone looking to become a Test Manager or for anyone who has newly started the role.

Ensuring a Reliable Digital Experience This Black Friday

With so much data and so many combinations of state, digital shopping experiences can be challenging to test. Senior Director of Product Marketing Dan Giordano talks about how to test your eCommerce application to prioritize coverage on the most important parts of your application. He also shares some common shopper personas to help you start putting together your own user scenarios. The live demo shows how AI-powered automated visual testing can help retail businesses in the areas of visual regression testing, accessibility testing, and multi-baseline testing for A/B experiments.

Dan gave a recap and went a little deeper into eCommerce testing in a follow-up blog post.

Cypress, Playwright, Selenium, or WebdriverIO? Let the Engineers Speak!

Our popular Let the Code Speak webinar series focused primarily on differences in syntax and features, but it doesn’t really cover how these frameworks hold up in the long term. In our new Let the Engineers Speak webinar, we spoke with a panel of engineers from Mercari US, YOOBIC, Hilton, Q2, and Domino’s about how they use Cypress, Playwright, Selenium, and WebdriverIO in their day-to-day operations. Andrew Knight moderated as our panelists discussed what challenges they faced and if they ever switched from one framework to another. The webinar gives a great view into the factors that go into deciding what tool is right for the project.

Read the Let the Engineers Speak recap blog series

Let the Engineers Speak! Part 5: Audience Q&A

Articles — 01.11.2023

Let the Engineers Speak! Part 4: Changing Frameworks

Articles — 01.05.2023

Let the Engineers Speak! Part 3: Favorite Test Integrations

Articles — 12.28.2022

Let the Engineers Speak! Part 2: The Biggest Challenges

Articles — 12.22.2022

More on the way in 2023!

We’ve got even more great test automation content coming this year. Be sure to visit our upcoming events page to see what we have lined up.

Check out our on-demand video library for all of our past videos. If you have any favorite videos from this list or from 2022, you can let us know @Applitools. Happy testing!

WebdriverIO is an amazing test automation framework, and I am very excited that on December 1, 2022 WebdriverIO officially announced its version 8 release! With its previous updates in versions 6 and 7, the WebdriverIO team rewrote everything in TypeScript, removed sync support to transition to async-only, and added integration for Google Lighthouse. This version 8 release is a welcomed refinement of the framework. It shouldn’t change much for end users, nor should it have breaking changes.

With its update to version 8, WebdriverIO: 

• updated the helpful one-line command to set up a new project
• removes support for older versions of Node.js
• transitions from CommonJS to ESM
• implements a new Action API interface
• adds a new runner for Unit and Component Testing in the browser
• streamlines the way WebdriverIO deals with global objects using the test runner

Of course, the team also updated the documentation, too.

Let’s explore the WebdriverIO v8 updates.

Update to One-line Setup Command

A small but effective update so far that made me smile was being able to set up a WebdriverIO project with one command `$ npm init wdio@latest ./`. After that command, it asks you all the questions needed to set up and run your tests. It provides you with predefined options for not only end-to-end testing of web and mobile applications but unit and component testing as well.

Support for Node.js v12, v13, and v14 dropped

The Node.js team moved Node.js v14 into a maintenance Long Term Support (LTS) phase in October 2021. It is now in “maintenance” until its end-of-life in April 2023. With newer versions of Node.js available, the WebdriverIO team recommends updating to Node.js v16 or v18 directly.

Find out more about Node.js release schedule.

Transition from Common JS to EMCAScript modules (ESM)

To give some background, CommonJS and ESM refer to the module system that allows us to use code from a “next” developer/library or code/data that is in a different file. The CommonJS module system has been the default module system within the Node.js ecosystem. However, the ES module was added in Node.js v8.5.0 and has been stabilized and incorporated as standard from Node.js v13.2.0. 
You may be familiar with using `require` to use your page objects in your test files and `module.exports` or `exports` so that your page objects or other files can be accessed by your tests. This method of managing files and libraries is the CommonJS way.

With ESM you use `import` and `export` to manage files and libraries.

There are some other differences between them, but for you, the end user, this is the main one. 
This change should not affect your work as CommonJS is still supported. However, if you want to use ES modules then you can now do so. I recommend getting familiar with ESM and how it works before starting to use it. Hopefully, in the near future, there will be additional boilerplates from WebdriverIO on using ESM in your projects https://webdriver.io/docs/boilerplates/.

New Runner for Unit and Component Testing in the Browser

An exciting v8 feature is the new browser runner that allows you to run unit and component tests in an actual/real browser environment. This is going to provide you with a more realistic user interaction with your components. It comes with presets for React, Vue, Svelte, SolidJS among others, and is powered by Vite.js.

Find out more details about this new runner.

New Action Interface

This is a new interface that makes executing various actions much easier. Two new browser commands action and actions have been added. It is now much simpler and type-safe to run the right action, e.g. sending key events to the browser.

Read more about this in the WebdriverIO API docs.

WebDriver BiDi Support

WebdriverIO is based on the WebDriver protocol, which is a web standard for automating browsers. This means that, with WebdriverIO, your tests are running in a browser that your user uses rather than a browser engine. 

This WebDriver BiDi Support means that whenever new protocol changes happen in Webdriver, you as the user can use it once it’s available in browsers. 
The idea of WebDriver BiDi is to make a new standard protocol for browser automation, which will be based on the bi-directional transport protocol (WebSocket or pipes).

Optional Globals

The WebdriverIO test runner would usually register the browser object or the $ and $$ commands to the global scope. With v8 update, the user can decide if they want to continue attaching these objects and methods to the global scope or prefer importing them directly.

To import them directly you would say : 

import { browser, $, $$, expect } from ‘@wdio/globals’

Importing the $ command allows you to use it from the @wdio/globals:

Using Visual Studio Code as my code editor, if I hover over the $ it shows that it is coming as a global import from WebdriverIO:

A new configuration property called injectGlobals (defaults: true) handles whether the test runner modifies the global scope or not. If your setup works fine using global objects, no change is needed to update to v8.

You don’t have to import from @wdio/globals as it will still work.

Other Updates

Aside from these major updates, documentation has been improved and new API docs around WebdriverIO objects like browser, element, and mock have been introduced. The behavior of relative spec or exclude paths was also fixed so that now specs and exclude paths will be always seen as relative to the config file and –spec arguments, relative from the working directory.
To see all of the details of this latest release, visit the WebdriverIO blog.

Update to WebdriverIO version 8 from version 7

Install a module to help update outdated npm modules

`npm install -g npm-check-updates`

Run `ncu` to show list of outdated npm packages:

Then run `ncu -u` to upgrade your package.json:

Then finally run npm install to install new versions:

I then reran all my tests and they passed successfully.

My thoughts on the version update

WebdriverIO’s version 8 update is a continuation of work by the WebdriverIO team to make the framework more rounded and robust. I like the changes that have been made and I am looking forward to trying out unit component testing with WebdriverIO.

I love that the updates don’t have breaking changes and so this allowed me to upgrade from version 7 to 8. I would recommend that you update your project to the latest version to benefit from the updates that have been made.
If you are interested in learning more about WebdriverIO, check out my WebdriverIO course on Test Automation University. Also, check out this repo with a sample example using WebdriverIO v8.