Cypress Archives - Automated Visual Testing | Applitools

Let the Engineers Speak! Part 4: Changing Frameworks

Let the Engineers Speak! Part 3: Favorite Test Integrations

The post Future-Proofing Your Test Automation Pipeline appeared first on Automated Visual Testing | Applitools.

Using TypeScript for Test Automation

Filip Hric — Wed, 18 Jan 2023 21:25:08 +0000

TypeScript is not only for developers anymore. If you are working as a tester in a web development team, chances are that you have heard about TypeScript. It’s been getting more and more attention over the past couple of years and has even surpassed JavaScript as the dominant language in a recent survey on state of software delivery made by CircleCI.

In my very un-scientific poll on LinkedIn, I asked fellow web application testers about the programming language of their choice. It seems that JavaScript is the most popular choice, winning over TypeScript and Java. But, in my opinion, testers should pay attention to the rising popularity of TypeScript and ideally start using it. In this article, I would like to take a closer look at many of TypeScript’s benefits for testing and automation.

What is TypeScript?

TypeScript is a programming language that is a superset of JavaScript. It adds many extra capabilities to JavaScript, improving the overall developer experience. As Basarat Ali Syed aptly puts it, TypeScript gives you the ability to use future JavaScript today. Besides that, TypeScript adds a type system into JavaScript, helping you write more stable and maintainable code. Let me give you an example of what that means.

Look at this plain JavaScript function:

const addition = (a, b) => {
 return a + b
}

This function takes two parameters and adds them up. It’s helpful if we need to sum two numbers. But what if we use this function in a way it was not intended? Or worse – what if we misunderstood how this function works?

addition(1, 2) // returns 3
addition('1', '2') // returns '12'

In the example above, our function will yield different results based on the type of input we provide it with. On the first line, we are passing two numbers, and our function will correctly add them up. On the second line, we are passing two strings, and since the input values are wrapped in quotation marks, the function will concatenate them instead of adding the numbers.

The function works as designed, but even if that’s the case, we may get into unexpected results. After all, this is why software testing is a thing.

But as developers work on more complex projects, on more complex problems, and with more complex data, risks increase. This is where TypeScript can become very helpful, since it can specify what kind of input the function expects. Let’s see how a similar function definition and function call would look like in TypeScript:

const addition = (a: number, b: number) => {
 return a + b
}


addition(1, 2) // returns 3
addition('1', '2') // shows an error

In the function definition, we specify the types for the parameters that this function expects. If we try to use our add() function incorrectly, the TypeScript compiler will complain about this and throw an error. So what is TypeScript compiler, you ask?

How TypeScript works

TypeScript cannot be read by the browser. In order to run TypeScript code, it needs to be compiled. In other words, everything you create using TypeScript will be converted to JavaScript at some point.

To run the compiler, we can open the terminal and run the following command:

tsc addition.ts

This command will point TypeScript compiler (tsc) to our addition.ts file. It will create a JavaScript file alongside the original TypeScript file. If there are any “type errors” in the file, we’ll get an error into our terminal.

But you don’t have to do this manually every time. There’s a good chance your code editor has a TypeScript compiler running in the background as you type your code. With VS Code, this functionality comes out of the box, which makes sense since both TypeScript and VS Code are developed and maintained by Microsoft. Having the compiler running in the background is incredibly useful, mostly because this allows us to immediately see errors such as the one shown in the last example. Whenever there is such an error, we get feedback and an explanation:

In this case, we are passing a string into a function that requires us to pass numbers. This information comes from a compiler that runs inside the editor (VS code in my case).

Additionally, some modern testing tools such as Playwright and Cypress run the compiler on the fly and convert your TypeScript code into browser-readable JavaScript for you.

How to use TypeScript as a tester

Now that you know what TypeScript is and how it works, it is time to answer the most important question: why? Is TypeScript even useful for someone who focuses on test automation?

My answer is yes, and I would like to demonstrate this in a few examples. I’ll also give you a couple of reasons why I think test automation engineers should start getting familiar with TypeScript.

Typed libraries

As test automation engineers, we often implement many different libraries for our work. There’s no tool that fits all the needs, and many times we deal with plugins, integrations, toolsets, extensions, and libraries. When you start, you need to get yourself familiar with the API of that tool, dig into the documentation, try it out, and understand different commands. It’s a process. And it can be exhausting to get through that first mile.

TypeScript can be helpful in speeding up that process. Libraries that contain type definitions can really get you up to speed with using them. When a library or a plugin contains type definitions, it means that functions in that library will have the same type of checking implemented as in the example I have given earlier.

This means that whenever you e.g. pass a wrong argument to a function, you will see an error in your editor. In the following example, I am passing a number into a cy.type() function that will only accept text:

Code autocompletion

Besides checking for correct arguments, TypeScript can help with giving autocomplete suggestions. This can act as a quick search tool, when you are looking for the right command or argument.

I have recently made a plugin for testing API with Cypress, and TypeScript autocompletion helps with passing different attributes such as url, method, or request body:

Handling imported data

Working with data can often get complicated. Especially when working with complex datasets in which you have to navigate through multiple levels of data structure. One mistake can make the whole test fail and cause a headache when trying to debug that problem.

When data is imported to a test, for example from a JSON file, the structure of that JSON file is imported to the test as well. This is something that the TypeScript compiler inside the editor does for us. Let’s take a look at a simple JSON file that will seed data into our test:

While creating our test, the editor will guide us through the fixture file and suggest possible keys that can be inferred from that file:

Notice how we not only get the key names, but also the type of the key. We can use this type inference for both seeding the data into our tests, as well as making assertions. The best part about this? The fixture file and test are now interconnected. Whenever we change our fixture file, the test file will be affected as well. If e.g. we decide to change the name property in our JSON file to title, TypeScript compiler will notice this error even before we decide to run our test.

Tightening source code with test code

Probably my strongest argument for using TypeScript is the connection between test code and source code. Being able to tie things together is a game changer. Whenever the source code changes, it can have a direct effect on tests, and with TypeScript, there’s a high chance it will show even before you run your tests on a pipeline.

Let’s say you have an API test. If your developers use TypeScript, chances are they have a TypeScript definition of the API structure. As a tester, you can import that structure into your test and use it e.g. for testing the API response:

import Board from "trelloapp/src/typings/board";


it('Returns proper response when creating new board', () => {
 cy.request('POST', '/api/boards', { name })
   .then(({ body }) => {
     expect(body.name).to.eq(name)
     expect(body.id).to.exist
   })
 })

Similarly to the previous example with fixture files, whenever something changes in our typings file, we will notice the change in the test file as well.

Checking errors on CLI

A really powerful feature of TypeScript is the ability to check all errors in the project. We can do this by typing following command in the terminal:

tsc --noEmit

The –noEmit flag means that the TypeScript compiler will not create JavaScript files, but it will check for any errors on our files. We can check all our files in the project, which means that even if we have worked on a single file, all the files will be checked for errors.

We can check the health of our tests even before they are run. Whenever we change files in our project, we can check if type checks are still passing. Even without opening any files.

This is sometimes referred to as “static testing”. It enables us to add an additional layer of checks that will help us make less mistakes in our code.

A great advantage of this is that it runs super fast, making it a good candidate for a pre-commit check.

In fact, setting up such a check is very easy and can be done in three simple steps:

First, install pre-commit package via npm using following command:

npm install pre-commit --save-dev

Then, create a lint script in package.json:

"scripts": {
 "lint": "tsc --noEmit"
}

As a final step, define lint as one of the pre-commit checks in package.json:

"pre-commit": [ "lint" ]

From now on, whenever we try to commit, our tsc –noEmit script will run. If it throws any errors, we will not be able to commit staged files.

Conclusion

TypeScript offers a variety of advantages. Static typing can improve the reliability and maintainability of the test code. It can help catch errors and issues earlier in the development process, saving time and effort spent on debugging. And there are many more that didn’t make it to this post.

Since TypeScript is built on top of JavaScript, test automation engineers who are familiar with JavaScript will be able to easily pick up TypeScript. The knowledge and skills they have developed in JavaScript will transfer over. If you need the initial push, you can check out my new course on TypeScript in Cypress, where I explain the basics of TypeScript within Cypress, but most of the knowledge can be transferred to other tools as well. The best part? It’s absolutely free on Test Automation University.

The post Using TypeScript for Test Automation appeared first on Automated Visual Testing | Applitools.

Let the Engineers Speak! Part 5: Audience Q&A

Andrew Knight — Wed, 11 Jan 2023 14:58:00 +0000

In this final part of our Cypress, Playwright, Selenium, or WebdriverIO? Let The Engineers Speak recap series, we will cover the audience Q&A, sharing the most popular questions from the audience and the answers our experts gave. Be sure to read our previous post.

The experts

I’m Andrew Knight – the Automation Panda – and I moderated this panel. Here were the panelists and the frameworks they represented:

Gleb Bahmutov (Cypress) – Senior Director of Engineering at Mercari US
Carter Capocaccia (Cypress) – Senior Engineering Manager – Quality Automation at Hilton
Tally Barak (Playwright) – Software Architect at YOOBIC
Steve Hernandez (Selenium) – Software Engineer in Test at Q2
Jose Morales (WebdriverIO) – Automation Architect at Domino’s

The discussion

Andy Knight (moderator):

So our first question comes from Jonathan Nathan.

Can Playwright or Cypress handle multiple browser tabs? What do engineers in these tools do for Azure authentication or target new links?

Gleb Bahmutov (Cypress):

[It’s a] very specific kind of interesting question, right? Multiple tabs and Azure authentication. You must have both. I think Playwright is [a] better tool, because it supports multiple browser tabs control right out of the box. So I would go with that.

Carter Capocaccia (Cypress):

Can I share a hack for this real quick?

Andy Knight (moderator):

Sure.

Carter Capocaccia (Cypress):

So I’m going to share the way that you don’t have to deal with multiple browser tabs in Cypress. And that’s just, you change the DOM from a target blank to a target self and then it just opens in the same window. So if you have to use Cypress [and] you need to get around multiple browser tabs, you can do DOM manipulation with Cypress.

Andy Knight (moderator):

Man, it sounds kind of hacky, though.

Carter Capocaccia (Cypress):

No, I mean it’s absolutely right. Yeah, it’s hacky.

Andy Knight (moderator):

So I got a question then. Is that like a feature request anyone has lifted up for Cypress? Or is that just because of the way the Cypress app itself is designed that that’s just a non-starter?

Gleb Bahmutov (Cypress):

It’s an open issue. Cypress team says it’s not a priority. Because you think about two tabs, right? Two tabs that present communication between one user and another of a backend. So you always want to kind of stop that and control it so you don’t have to actually have two windows. You want to control one window and communicate with API calls on the back. At least that’s the Cypress team opinion. So we might not see it any time soon.

ICYMI: Cypress ambassador Filip Hric shared What’s New in Cypress 12, which includes an update on the feature just discussed.

Andy Knight (moderator):

Yeah. I know in Playwright like, Gleb, like you said, it is really easy and nice because you have a browser instance. Then in that instance, you have multiple browser contexts and then, from each browser context, you’re going to have multiple pages. So yeah, I love it. Love it.

Tally Barak (Playwright):

And the better thing is that you can have one of them, like in a mobile size or emulating a mobile device and the other one in web. So if you want to run a test of like cutting between two different users, each one is incognito and is a complete browser context. They don’t share their local storage or anything. So basically, you can run any test that you want on the thing. And because of the browser, it also works really fast. Because it’s not really launching a whole browser. It’s just launching a browser context, which is much, much faster than launching the whole browser.

Andy Knight (moderator):

Awesome. Alright, let’s move on to our next question here. This is from Sundakar.

Many of the customers I worked with are preferring open source. And do you think Applitools will have its place in this open source market?

Can I answer this one? Because I work for Applitools.

For this one, I think absolutely yes. I mean all of Applitools Eyes SDKs are open source. What Applitools provides is not only just the mechanism for visual testing, but also the platforms. We work with all the open source tools, frameworks, you name it. So absolutely, I would say there’s a place here. Let me move on to the next question.

Gleb Bahmutov (Cypress):

Andy, before you move on, can I add? So my computer science PhD is in computer vision image processing. So it’s all about comparing new images, teaching them, and so on. I would not run my own visual testing service, right? My goal is to make sure I’m testing [the] web application. Keeping images, comparing them, showing the diffs, updating it. It’s such a hassle, it’s not worth it, my time. Just pay for a service like Applitools and move on with your life.

Andy Knight (moderator):

Awesome. Thank you. Okay. Let me pull the next question here. This is from Daniel.

I heard a lot that Playwright is still a new framework with a small community even when it was released in January of 2020 but never heard that about WebdriverIO. As far as I know, Playwright is older.

I don’t think that is true. I’d have to double check.

Tally Barak (Playwright):

No, I don’t think [so].

Andy Knight (moderator):

Is Playwright still considered new?

Tally Barak (Playwright):

It’s newer than the others. But it’s growing really fast. I mean, because I’m the [Playwright] OG, I remember the time when I would mention Playwright and no one had any idea what I’m talking about. It was very, very new. This is not the case anymore. I mean, there’s still, of course, people don’t really hear about it, but the community has grown a lot. I think [it has] over 40,000 stars on GitHub. The Slack channel has almost 5,000 participants or something. So the community is growing, Playwright is growing really, really nicely. And you’re welcome to join.

Andy Knight (moderator):

Okay, here’s another question from Ryan Barnes.

Do y’all integrate automated tests with a test case management tool? If so, which ones?

Gleb Bahmutov (Cypress):

Yes. TestRail.

Andy Knight (moderator):

TestRail. Okay.

Gleb Bahmutov (Cypress):

Because we are not the only testing tool, right? Across organizations, where our teams, our tools, and manual testing. So we need a central testing portal.

Tally Barak (Playwright):

No, we are not perfect. And we should. Any good ideas are welcome.

Carter Capocaccia (Cypress):

So we don’t have a formalized test manager tool. But if anybody has ever used any kind of Atlassian tooling – there’s, you know, JIRA out there has the idea of a test set ticket inside of the test set or individual test. You can define user flows inside of there. So I guess you can consider that a test management tool. It’s a little bit less featured than something like TestRail. Actually, it’s a lot less featured than something like TestRail. But you know, that’s how we stay organized. So we basically tie our tests to a ticket. That’s how we can manage, you know, well what is this ticket test? What is it supposed to be testing? Where is our source of truth?

Andy Knight (moderator):

I guess I could launch a somewhat controversial question here, but I’ll do it rhetorically not to answer. But if you have a test automation solution, do you really need to have it export results to a test case management tool? Or can you just use the reports it gives you? We’ll leave that for offline. So the next one on the list here is from Sindhuja.

We are trying to convert our test scripts from Protractor.

Okay. Wow, that’s a blast from the past.

We are doing [a] proof of concept and WebdriverIO and we have issues with running in Firefox and WebdriverIO. Is there any notes in WebdriverIO with cross browsers?

Jose, got any insights for us here?

Jose Morales (WebdriverIO):

Yeah, absolutely. So one thing that I really love about WebdriverIO is the easy configuration. So, when you create a project in WebdriverIO, you have a JSON file where you put all the configuration about capability services, what browser you want to use. And you can easily add your own configuration. It could be, for example, if you want to run in Firefox or in Edge or you want to run on Source Labs, you have several options.
So it is really easy to integrate configuration for Firefox. You only need to specify the browser in the capability section along with the version and special features like size view. If you want to know how to do that, it’s very easy. You can go to my home page. And there [are] examples where you can build something from scratch and you can see easily where to add that particular configuration. And I’m going to share with you some repositories in GitHub where you can see some examples [of] how to do it.

Andy Knight (moderator):

Thank you so much, Jose. Oh, here we go.

Which framework would be the best platform to test both Android and iOS apps?

I know most of us [are] focused on web UI, so here’s a curveball: mobile.

Gleb Bahmutov (Cypress):

I can say that at Mercari US, we picked Detox after using Appium for a long time. But for new React Native projects, we went with Detox.

Carter Capocaccia (Cypress):

Yeah, Detox is the only one that I’ve ever used it for as well. And it was really, really good. I found no reason to switch. I think, Gleb, can you correct me if I’m wrong on this? I think Detox was originally made by Wix? Is it, Wix, the company?

Gleb Bahmutov (Cypress):

That’s correct. Yes.

Carter Capocaccia (Cypress):

Yes, so Wix used to make Detox. I think it’s still maintained by them, but it was like an in-house tool they open sourced, and now it’s really good.

Andy Knight (moderator):

Awesome. Cool. I hadn’t heard. I’ll have to look it up. Alrighty, well I think that’s about all the time we have for question and answer today. I want to say thank you to everyone for attending. Thank you to all of our speakers here on the panel.

Conclusion

This article concludes our Cypress, Playwright, Selenium, or WebdriverIO? Let The Engineers Speak recap series. We got to hear from engineers at Mercari, YOOBIC, Hilton, Q2, and Domino’s about how their teams build their test automation projects and why they made their framework decisions. Our panelists also shared insights into advantages and disadvantages they’ve encountered in their test automation frameworks. If you missed any previous part of the series, be sure to check them out:

The post Let the Engineers Speak! Part 5: Audience Q&A appeared first on Automated Visual Testing | Applitools.

What’s New in Cypress 12

Filip Hric — Tue, 10 Jan 2023 17:56:27 +0000

Right before the end of 2022, Cypress surprised us with their new major release: version 12. There wasn’t too much talk around it, but in terms of developer experience (DX), it’s arguably one of their best releases of the year. It removes some of the biggest friction points, adds new features, and provides better stability for your tests. Let’s break down the most significant ones and talk about why they matter.

No more “detached from DOM” errors

If you are a daily Cypress user, chances are you have seen an error that said something like, “the element was detached from DOM”. This is often caused by the fact that the element you tried to select was re-rendered, disappeared, or detached some other way. With modern web applications, this is something that happens quite often. Cypress could deal with this reasonably well, but the API was not intuitive enough. In fact, I listed this as one of the most common mistakes in my talk earlier this year.

Let’s consider the example from my talk. In a test, we want to do the following:

Open the search box.
Type “abc” into the search box.
Verify that the first result is an item with the text “abc”.

As we type into the search box, an HTTP request is sent with every keystroke. Every response from that HTTP request then triggers re-rendering of the results.

The test will look like this:

it('Searching for item with the text "abc"', () => {
 
 cy.visit('/')
 
 cy.realPress(['Meta', 'k'])
 
 cy.get('[data-cy=search-input]')
   .type('abc')
 
 cy.get('[data-cy=result-item]')
   .first()
   .should('contain.text', 'abc')
 
})

The main problem here is that we ignore the HTTP requests that re-render our results. Depending on the moment when we call cy.get() and cy.first() commands, we get different results. As the server responds with search results (different with each keystroke), our DOM is getting re-rendered, making our “abc” item shift from second position to first. This means that our cy.should() command might make an assertion on a different element than we expect.

Typically, we rely on Cypress’ built-in retry-ability to do the trick. The only problem is that the cy.should() command will retry itself and the previous command, but it will not climb up the command chain to the cy.get() command.

It is fairly easy to solve this problem in versions v11 and before, but the newest Cypress update has brought much more clarity to the whole flow. Instead of the cy.should() command retrying only itself and the previous command, it will retry the whole chain, including our cy.get() command from the example.

In order to keep retry-ability sensible, Cypress team has split commands into three categories:

assertions
actions
queries

These categories are reflected in Cypress documentation. The fundamental principle brought by version 12 is that a chain of queries is retried as a whole, instead of just the last and penultimate command. This is best demonstrated by an example comparing versions:

// Cypress v11:
cy.get('[data-cy=result-item]') //  not retried
 .first() // retried
 .should('contain.text', 'abc') // retried
 
// Cypress v12:
cy.get('[data-cy=result-item]') //  retried
 .first() // retried
 .should('contain.text', 'abc') // retried

cy.get() and cy.first() are commands that both fall into queries category, which means that they are going to get retried when cy.should() does not pass immediately. As always, Cypress is going to keep on retrying until the assertion passes or until a time limit runs up.

cy.session() and cy.origin() are out of beta

One of the biggest criticisms of Cypress.io has been the limited ability to visit multiple domains during a test. This is a huge blocker for many test automation engineers, especially if you need to use a third-party domain to authenticate into your application.

Cypress has advised to use programmatic login and to generally avoid trying to test applications you are not in control of. While these are good advice, it is much harder to execute them in real life, especially when you are in a hurry to get a good testing coverage. It is much easier (and more intuitive) to navigate your app like a real user and automate a flow similar to their behavior.

This is why it seems so odd that it took so long for Cypress to implement the ability to navigate through multiple domains. The reason for this is actually rooted in how Cypress is designed. Instead of calling browser actions the same way as tools like Playwright and Selenium do, Cypress inserts the test script right inside the browser and automates actions from within. There are two iframes, one for the script and one for the application under test. Because of this design, browser security rules limit how these iframes interact and navigate. Laying grounds for solving these limitations were actually present in earlier Cypress releases and have finally landed in full with version 12 release. If you want to read more about this, you should check out Cypress’ official blog on this topic – it’s an excellent read.

There are still some specifics on how to navigate to a third party domain in Cypress, best shown by an example:

it('Google SSO login', () => {
 
 cy.visit('/login') // primary app login page
 
 cy.getDataCy('google-button')
   .click() // clicking the button will redirect to another domain
 
 cy.origin('https://accounts.google.com', () => {
   cy.get('[type="email"]')
     .type(Cypress.env('email')) // google email
   cy.get('[type="button"]')
     .click()
   cy.get('[type="password"]')
     .type(Cypress.env('password')) // google password
   cy.get('[type="button"]')
     .click()
 })
 
 cy.location('pathname')
   .should('eq', '/success') // check that we have successfully
 
})

As you see, all the actions that belong to another domain are wrapped in the callback of cy.origin() command. This separates actions that happen on the third party domain.

The Cypress team actually developed this feature alongside another one that came out from beta, cy.session(). This command makes authenticating in your end-to-end tests much more effective. Instead of logging in before every test, you can log in just once, cache that login, and re-use it across all your specs. I recently wrote a walkthrough of this command on my blog and showed how you can use it instead of a classic page object.

This command is especially useful for the use case from the previous code example. Third-party login services usually have security measures in place that prevent bots or automated scripts from trying to login too often. If you attempt to login too many times, you might get hit with CAPTCHA or some other rate-limiting feature. This is definitely a risk when running tens or hundreds of tests.

it('Google SSO login', () => {
 
 cy.visit('/login') // primary app login page
 cy.getDataCy('google-button')
   .click() // clicking the button will redirect to another domain
 
 cy.session('google login', () => {
   cy.origin('https://accounts.google.com', () => {
     cy.get('[type="email"]')
       .type(Cypress.env('email')) // google email
     cy.get('[type="button"]')
       .click()
     cy.get('[type="password"]')
       .type(Cypress.env('password')) // google password
     cy.get('[type="button"]')
       .click()
   })
 })
 
 cy.location('pathname')
   .should('eq', '/success') // check that we have successfully
 
})

When running a test, Cypress will make a decision when it reaches the cy.session() command:

Is there a session called google login anywhere in the test suite?
- If not, run the commands inside the callback and cache the cookies, local storage, and other browser data.
- If yes, restore the cache assigned to a session called “google login.”

You can create multiple of these sessions and test your application using different accounts. This is useful if you want to test different account privileges or just see how the application behaves when seen by different accounts. Instead of going through the login sequence through UI or trying to log in programmatically, you can quickly restore the session and reuse it across all your tests.

This also means that you will reduce your login attempts to a minimum and prevent getting rate-limited on your third party login service.

Run all specs in GUI

Cypress GUI is a great companion for writing and debugging your tests. With the version 10 release, it has dropped support for the “Run all specs” button in the GUI. The community was not very happy about this change, so Cypress decided to bring it back.

The reason why it was removed in the first place is that it could bring some unexpected results. Simply put, this functionality would merge all your tests into one single file. This can get tricky especially if you use before(), beforeEach(), after() and afterEach() hooks in your tests. These would often get ordered and stacked in unexpected order. Take following example:

// file #1
describe('group 1', () => {
 it('test A', () => {
   // ...
 })
})
 
it('test B', () => {
 // ...
})
 
// file #2
before( () => {
 // ...
})
 
it('test C', () => {
 // ...
})

If this runs as a single file, the order of actions would go like this:

before() hook
test B
test C
test A

This is mainly caused by how Mocha framework executes blocks of code. If you properly wrap every test into describe() blocks, you would get much less surprises, but that’s not always what people do.

On the other hand, running all specs can be really useful when developing an application. I use this feature to get immediate feedback on changes I make in my code when I work on my cypress plugin for testing API. Whenever I make a change, all my tests re-run and I can see all the bugs that I’ve introduced. ?

Running all specs is now behind an experimental flag, so you need to set experimentalRunAllSpecs to true in your cypress.config.js configuration file.

Test isolation

It is always a good idea to keep your tests isolated. If your tests depend on one another, it may create a domino effect. First test will make all the subsequent tests fail as well. Things get even more hairy when you bring parallelisation into the equation.

You could say that Cypress is an opinionated testing framework, but my personal take on this is that this is a good opinion to have. The way Cypress enforces test isolation with this update is simple. In between every test, Cypress will navigate from your application to a blank page. So in addition to all the cleaning up Cypress did before (clearing cookies, local storage), it will now make sure to “restart” the tested application as well.

In practice the test execution would look something like this:

it('test A', () => {
 cy.visit('https://staging.myapp.com')
 // ...
 // your test doing stuff
})
 
// navigates to about:blank
 
it('test B', () => {
 cy.get('#myElement') // nope, will fail, we are at about:blank
})

This behavior is configurable, so if you need some time to adjust to this change, you can set testIsolation to false in your configuration.

Removing of deprecated commands and APIs

Some of the APIs and commands reached end of life with the latest Cypress release. For example, cy.route() and cy.server() have been replaced by the much more powerful cy.intercept() command that was introduced back in version 6.

The more impactful change was the deprecation of Cypress.Cookies.default() and Cypress.Cookies.preserveOnce() APIs that were used for handling the behavior of clearing up and preserving cookies. With the introduction of cy.session(), these APIs didn’t fit well into the system. The migration from these commands to cy.session() might not seem as straightforward, but it is quite simple when you look at it.

For example, instead of using Cypress.Cookies.preserveOnce() function to prevent deletion of certain cookies you can use cy.session() like this:

beforeEach(() => {
 cy.session('importantCookies', () => {
   cy.setCookie('authentication', 'top_secret');
 })
});
 
it('test A', () => {
 cy.visit('/');
});
 
it('test B', () => {
 cy.visit('/');
});

Also, instead of using Cypress.Cookies.defaults() to set up default cookies for your tests, you can go to your cypress/support/e2e.js support file and set up a global beforeEach() hook that will do the same as shown in the previous example.

Besides these there were a couple of bug fixes and smaller tweaks which can all be viewed in Cypress changelog. Overall, I think that the v12 release of Cypress is one of the unsung heroes. Rewriting of query commands and availability of cy.session() and cy.origin() commands may not seem like a big deal on paper, but it will make the experience much smoother than it was before.

New command queries might require some rewriting in your tests. But I would advise you to upgrade as soon as possible, as this update will bring much more stability to your tests. I’d also advise to rethink your test suite and integrate cy.session() to your tests as it might not only handle your login actions more elegantly but shave off minutes of your test run.

If you want to learn more about Cypress, you can come visit my blog, subscribe to my YouTube channel, or connect with me on Twitter or LinkedIn.

Let the Engineers Speak! Part 5: Audience Q&A

Let the Engineers Speak! Part 4: Changing Frameworks

Let the Engineers Speak! Part 3: Favorite Test Integrations

The post What’s New in Cypress 12 appeared first on Automated Visual Testing | Applitools.

Test Automation Video Winter Roundup: September – December 2022

Applitools Team — Mon, 09 Jan 2023 18:35:00 +0000

Check out the latest test automation videos from Applitools.

We hope you got to take time to rest, unplug, and spend time with your loved ones to finish out 2022 with gratitude. I have been incredibly appreciative of the learning opportunities and personal growth that 2022 offered. In reflection of our past quarter here at Applitools, we’ve curated our latest videos from some amazing speakers. If you missed any videos while away on holiday or finishing off tasks for the year, we’ve gathered the highlights for you in one spot.

ICYMI: Back in November, Andrew Knight (a.k.a. the Automation Panda) shared the top ten Test Automation University courses.

Cypress vs. Playwright: The Rematch

One of our most popular series is Let the Code Speak, where we compare testing frameworks in real examples. In our rematch of Let the Code Speak: Cypress vs. Playwright, Andrew Knight and Filip Hric dive deeper to how Cypress and Playwright work in practical projects. Quality Engineer Beth Marshall moderates this battle of testing frameworks while Andy and Filip explore comparisons of their respective testing frameworks in the areas of developer experience, finding selectors, reporting, and more.

Automating Testing in a Component Library

Visual testing components allows teams to find bugs earlier, across a variety of browsers and viewports, by testing reused components in isolation. Software Engineering Manager David Lindley and Senior Software Engineer Ben Hudson joined us last year to detail how Vodafone introduced Applitools into its workflow to automate visual component testing. They also share the challenges and improvements they saw when automating their component testing.

When to Shift Left, Move to Centre, and Go Right in Testing

Quality isn’t limited to the end of the development process, so testing should be kept in mind long before your app is built. Quality Advocate Millan Kaul offers actionable strategies and answers to questions about how to approach testing during different development phases and when you should or shouldn’t automate. Millan also shares real examples of how to do performance and security testing.

You, Me, and Accessibility: Empathy and Human-Centered Design Thinking

Inclusive design makes it easier for your customers with your varying needs and devices are able to use your product. Accessibility Advocate and Crema Test Engineer Erin Hess talks about the principles of accessible design, how empathy empowers teams and end users, and how to make accessibility more approachable to teams that are newer to it. This webinar is helpful all team members, whether you’re a designer, developer, tester, product owner, or customer advocate.

Erin also shared a recap along with the audience poll results in a follow-up blog post.

Future of Testing October 2022

Our October Future of Testing event was full of experts from SenseIT, Studylog, Meta, This Dot, EVERSANA, EVERFI, LAB Group, and our own speakers from Applitools. We covered test automation topics across ROI measurement, accessibility, testing at scale, and more. Andrew Knight, Director of Test Automation University, concludes the event with eight testing convictions inspired by Ukiyo-e Japanese woodblock prints. Check out the full Future of Testing October 2022 event library for all of the sessions.

Skills and Strategies for New Test Managers

Being a good Test Manager is about more than just choosing the right tools for your team. EasyJet Test Manager Laveena Ramchandani shares what she has learned in her experience on how to succeed in QA leadership. Some of Laveena’s strategies include how to create a culture that values feedback and communication. This webinar is great for anyone looking to become a Test Manager or for anyone who has newly started the role.

Ensuring a Reliable Digital Experience This Black Friday

With so much data and so many combinations of state, digital shopping experiences can be challenging to test. Senior Director of Product Marketing Dan Giordano talks about how to test your eCommerce application to prioritize coverage on the most important parts of your application. He also shares some common shopper personas to help you start putting together your own user scenarios. The live demo shows how AI-powered automated visual testing can help retail businesses in the areas of visual regression testing, accessibility testing, and multi-baseline testing for A/B experiments.

Dan gave a recap and went a little deeper into eCommerce testing in a follow-up blog post.

Cypress, Playwright, Selenium, or WebdriverIO? Let the Engineers Speak!

Our popular Let the Code Speak webinar series focused primarily on differences in syntax and features, but it doesn’t really cover how these frameworks hold up in the long term. In our new Let the Engineers Speak webinar, we spoke with a panel of engineers from Mercari US, YOOBIC, Hilton, Q2, and Domino’s about how they use Cypress, Playwright, Selenium, and WebdriverIO in their day-to-day operations. Andrew Knight moderated as our panelists discussed what challenges they faced and if they ever switched from one framework to another. The webinar gives a great view into the factors that go into deciding what tool is right for the project.

Read the Let the Engineers Speak recap blog series

Let the Engineers Speak! Part 5: Audience Q&A

Let the Engineers Speak! Part 4: Changing Frameworks

Let the Engineers Speak! Part 3: Favorite Test Integrations

Let the Engineers Speak! Part 2: The Biggest Challenges

More on the way in 2023!

We’ve got even more great test automation content coming this year. Be sure to visit our upcoming events page to see what we have lined up.

Check out our on-demand video library for all of our past videos. If you have any favorite videos from this list or from 2022, you can let us know @Applitools. Happy testing!

The post Test Automation Video Winter Roundup: September – December 2022 appeared first on Automated Visual Testing | Applitools.

Let the Engineers Speak! Part 4: Changing Frameworks

Andrew Knight — Thu, 05 Jan 2023 22:51:57 +0000

In part 4 of our Cypress, Playwright, Selenium, or WebdriverIO? Let The Engineers Speak recap series, we will recap the stories our panelists have about changing their test frameworks – or not! Be sure to read our previous post where our panelists talked about their favorite test integrations.

The experts

I’m Andrew Knight – the Automation Panda – and I moderated this panel. Here were the panelists and the frameworks they represented:

Gleb Bahmutov (Cypress) – Senior Director of Engineering at Mercari US
Carter Capocaccia (Cypress) – Senior Engineering Manager – Quality Automation at Hilton
Tally Barak (Playwright) – Software Architect at YOOBIC
Steve Hernandez (Selenium) – Software Engineer in Test at Q2
Jose Morales (WebdriverIO) – Automation Architect at Domino’s

The discussion

Andy Knight (moderator):

So I’m going to ask another popcorn question here. And this one again comes with an audience poll. So if you’re in the audience, hop over to the polls and take a look here. Have you ever considered switching your test tool or framework? And if so, why? If so, why not?

Carter Capocaccia (Cypress):

Yeah, we considered it and did it. You know, I think you have to acknowledge it at a certain point that maybe what you’re doing isn’t working or the tool doesn’t fit your application or your architecture. And it’s a really hard decision to make, right? Because typically, by the time you get to that point, you’ve invested tens, hundreds, if not thousands of hours into a framework or an architecture. And making that switch is not an easy task.

Andy Knight (moderator):

What did you switch from and to?

Carter Capocaccia (Cypress):

Yeah, so [Selenium] WebDriver to Cypress. And you can already kind of see that architecturally they’re very, very different. But yeah, it was not an easy task, not an easy undertaking, but I think we had to admit to ourselves that WebDriver did not fit our needs. It didn’t operate in the way we wanted it to operate. At least for us, it was set up in a way that did not match our development cycles, did not match our UI devs’ patterns and processes.

So when we talk about, you know, kind of making a team of test engineers have additional help, well, one way to do that is [to align] your test automation practices with your UI dev practices. And so if we can have a JavaScript test framework that runs in a web app, that acts like a web app, that feels like a web app, well, we’ve now just added an entire team of UI developers that we’ve kind of made them testers, but we haven’t told them that yet. That’s kind of how that works. You adopt these people by just aligning with their tooling and all of a sudden they realize like, hey, I can do this too.

Andy Knight (moderator):

Mm-hmm. So I want to ask another question. Since you’re saying you used Selenium WebDriver before, in what language did you automate your test? Was that in JavaScript, just like first test, or was that in Java or Python or something else?

Carter Capocaccia (Cypress):

Yep, so it’s JavaScript. But when we look [at the] architecture of the app, right? So the architecture of WebDriver is very different than Cypress. [With] Cypress, you write a JavaScript function just like you write any other function. And most of the time, it’s going to operate just like you expect it to. I think that’s where, instead of them having to learn a WebDriver framework and what the execution order of things was and how the configurations were, Cypress is just pretty apparent in how it functions. When you look at like, well, how is the Cypress test executed? It uses Mocha under the hood as the test runner. So that’s just functions that are just kind of inside of describe and context and it blocks. So I think anybody that doesn’t know anything about testing but is a JavaScript developer can hop right into Cypress, look at it, and say, yeah, I think I know what’s going on here. And probably within the first 5 to 10 minutes, write an assertion or write a selector.

Andy Knight (moderator):

Awesome. That’s really cool. And so was that something that you and your team or org did recently? Was this like a couple years ago that you made this transition?

Carter Capocaccia (Cypress):

You know, I think I’d be lying to you if I told you the transition ever ends, right? I think it starts and you get to a point where you’ve scaled and you’ve gotten to where you’ve written a ton of tests on it, and then that’s great. But of course, it’s kind of the bell curve of testing. Like it starts really, really slow because you’re moving off the old framework, then you start to scale really quickly. You peak, and then at that point, you have these kind of outliers on the other side of the bell that are just running legacy applications or they just don’t want to move off of it. Or for whatever reason, they can’t move off of it. Let’s say they’re running like Node 10 still. It’s like okay, well, maybe we don’t want to invest the time there to move them off of it.

Yeah. So it’s a long journey to do a task like this. So it’s never done. By the time we’re finished, you know, there’s gonna be a new testing tool or Cypress will have a new version or something else will be new in there. You know, pipelines will [be through] some AI network at this point. Who knows? But I’m sure we’ll have to start the next task.

Andy Knight (moderator):

Sure. So how about others in our group of panelist speakers here? Have y’all ever transitioned from one framework or tool to another? Or in your current project, did you establish it and you were just super happy with what you chose? And tell us the reasons why.

Tally Barak (Playwright):

Yes. So, yeah, we also switched from WebdriverIO as I said at the beginning. We switched to Playwright, obviously. It was not that painful, to be honest, maybe because we kept the Cucumber side. So we had [fewer] scenarios than what we have today. So it was less to move. I think it went over a period [of] a few months. Unlike you, Carter, we actually moved completely away from WebdriverIO. There is no WebdriverIO, everything is with Playwright. But again, probably a smaller organization, so we were able to do that. And are we happy? I mean, [we are] thankful every single day for this switch.

Gleb Bahmutov (Cypress):

I can comment. At Mercari, we did switch from WebDriver in Python and mabl. So both were because the maintenance of tests and debugging of failed tests was [a] very hard problem, and everyone agreed that there is a problem. The transition was, again, faster than we expected, just because the previous tests did not work very well. So at some point we were like, well, we have 50 failing daily tests, right? It’s not like we can keep them. It’s not a sunken cost. It’s a cost of not working. So we have to recreate the test. But it was not that bad. So for us, the cost of debugging and maintaining the previous tool just became overwhelming, and we decided, let’s just do something else.

Jose Morales (WebdriverIO):

In my experience, we were using Selenium with JavaScript as a previous framework, and then we moved to WebdriverIO with JavaScript and Applitools. Because in Domino’s, we have different kind of products. We have another product that is called Next Generation of the stores. We’re using a reactive application there. This is also a website application. And we’re evaluating other tools for automation such as UiPath. And one thing that I like about UiPath is [it’s] a low-coding solution. That means you don’t need to really be a very proficient developer in order to create the scenarios or automate with UiPath, because it only [requires] drags and drops [to] create the scenarios. And it’s very easy to automate. And in the top of that, with UiPath, it’s a very enterprise solution.

We have different kinds of components like an orchestrator, we have a test manager, we have integration with Jenkins. We have robots that execute our test cases. So it’s an enterprise solution that we implemented for that particular project. And we have the same results as WebdriverIO with Applitools. We’re happy with the two solutions – the two frameworks – and we’re considering to remove WebdriverIO and Applitools and use UiPath for the main website for Domino’s. So I think in the future, the low-coding solution for automation is a way to go in the future. So definitely UiPath is a good candidate for us for our automation in the website.

Andy Knight (moderator):

How about you, Steve? I know you’re right now on a Selenium-based test project. Have you and your team thought about either moving to something else, or do you feel pretty good about the whole Selenium-ness?

Steve Hernandez (Selenium):

I think clearly there appears to be major speed-ups that you get from the JavaScript-based solutions. And I know you’ve evangelized a little bit about Playwright and then I’m hearing good things from others. I mean, we have a mature solution. We have 98% pass rates. You know, some of the limitation is more what our application can handle when we blast it with tests. I don’t know if your developers after each local build are tests running locally using some of these JavaScripts solutions. But I think one of the things that is appealing is the speed. And with Boa Constrictor 3, the roadmap is to – well, it has been modularized. So you can bring your own Rusty API client or potentially swap in Selenium and swap in something like a module for Playwright. So we would get that speed up but keep the same API that we’re using. So we’re toying with it, but if we can pull that off in Boa Constrictor, that would take care of that for us.

Andy Knight (moderator):

Yeah, man, definitely. So, I mean, it sounds like a big trend from a lot of folks has been, you know, step away from things like Selenium WebDriver to [a] more modern framework. You know, because Selenium WebDriver is pretty much just a low-level browser automator, which is still great. It’s still fine. It’s still awesome. But I mean, one of the things I personally love about Playwright – as well as things like Cypress – is that it gives you that nice feature richness around it to not only help you run your tests, but also help develop your tests.

Audience Q&A

So we had a fantastic conversation amongst each of our panel speakers about the considerations they had as their test projects grew and changed. We found that most of our panelists have had to make changes to their test frameworks used as their projects’ needs changed. In the next article, we’ll cover the most popular questions raised by our audience during the webinar.

Let the Engineers Speak! Part 5: Audience Q&A

Let the Engineers Speak! Part 4: Changing Frameworks

Let the Engineers Speak! Part 3: Favorite Test Integrations

Let the Engineers Speak! Part 2: The Biggest Challenges

The post Let the Engineers Speak! Part 4: Changing Frameworks appeared first on Automated Visual Testing | Applitools.

Let the Engineers Speak! Part 3: Favorite Test Integrations

Andrew Knight — Wed, 28 Dec 2022 16:06:28 +0000

In part 3 of our Cypress, Playwright, Selenium, or WebdriverIO? Let the Engineers Speak recap series, we will recap our panelists’ favorite integrations in their test automation projects, as well as share our audience’s favorite integrations. Be sure to read our previous post, where our panelists discussed their biggest challenges in test automation.

The experts

I’m Andrew Knight – the Automation Panda – and I moderated this panel. Here were the panelists and the frameworks they represented:

Gleb Bahmutov (Cypress) – Senior Director of Engineering at Mercari US
Carter Capocaccia (Cypress) – Senior Engineering Manager – Quality Automation at Hilton
Tally Barak (Playwright) – Software Architect at YOOBIC
Steve Hernandez (Selenium) – Software Engineer in Test at Q2
Jose Morales (WebdriverIO) – Automation Architect at Domino’s

The discussion

Andy Knight (moderator):

Alrighty. So I mean, this, our conversation is now kind of going in so many good directions. This is awesome. We keep talking about integrations and how our frameworks and our tools can connect with other things. So I guess my next question I would like to ask as a big popcorn question would be: what are some of your favorite integrations with your test projects? I’m talking about things like CI hooks, special reports that y’all have, maybe things for visual testing, accessibility testing.

And not only is this a question for the panel speakers, but also for the audience. We have another Slido word cloud, so head over to the Slido tab. We’d love to hear what your favorite integrations are for your test projects.

But I’d like to now direct this question back to our panel and ask them to pick one. Tell me why you love it so much. Tell me how y’all use it and what value y’all get out of it.

Carter Capocaccia (Cypress):

Okay, I’ll hop in here. I think for me with Cypress, it’s just under the hood, it’s just JavaScript, right? So most anywhere is able to hook into that immediately. So when we talk about integrations, let’s just use GitHub for example here. If you wanted a GitHub Actions pipeline, you can just go ahead and start doing that immediately. And Cypress out of the box will provide you a Docker image to go ahead and use right inside of your GitHub Actions pipeline.

So for me, I guess I’m not going to focus on any one specific tool for integration here. I think the way that Cypress kind of makes itself agnostic to all external tools is what makes it really, really nice. You know, it doesn’t say, hey, you’ve gotta use us with this particular, like an Azure pipeline or any kind of other DevOps tool. It says, hey, under the hood, all we are is a JavaScript framework. If you want to execute our JavaScript inside of some kind of Docker image, create your own Docker image. Gleb knows all about this. He publishes a new Docker image like every day it seems.

And we have, you know, the ability to take all that, take the results, and the base of what comes out of Cypress is just JSON. We’ll shoot that over to an open-source dashboard. So there’s Report Portal out there if you wanted to take those results, and then put them into a portal. I see on here [the poll results] Allure reports. Well, okay, so you take it, you just put it into any kind of integration there.
So I think for me personally – I actually wrote a whole blog post about this where I was able to take Vercel, GitHub Actions, and Cypress and set up a CI/CD pipeline completely for free with automated deployments, preview environments, automated testing, the whole nine yards. And so it made it to where for my personal little website, all I’ve got to do is submit a pull request. It kicks off all my tests. As soon as it’s done, it merges it, and then it’s immediately deployed with Vercel. So that was a really cool thing. So if I have to name a tool, I’m gonna say Vercel and GitHub Actions.

Gleb Bahmutov (Cypress):

Can I second that? GitHub Actions are amazing. Honestly, I went through the whole, you know, self-running Jenkins to TeamCity, Buildkite, Circle CI. GitHub Actions are the best. It’s amazing how much you can do with just JavaScript on UCI. And I was the one writing all integrations for the Cypress team at the time.

I want to bring one more integration. I’ve written it. It’s [an] integration between Cypress and LaunchDarkly feature flags environment, because during the test, sometimes you want to test an experiment behind a feature flag. And for us now, it’s an open-source plugin. It’s like this, here’s my user, here are the experiment values I want to enable. Let’s see this functionality, and it’s working.

Tally Barak (Playwright):

[My] answer is GitHub Actions as well. I don’t have that much experience derived with different tools. I used to work with Travis and a bit with CircleCI. But GitHub Actions is just answering everything we need – and Playwright obviously – because it just requires a node for the JavaScript version. You just install it with Docker, and you are good to go and do whatever you want with the outputs and so on. So yeah, GitHub actions and I’m actually interested in the other one that you mentioned about the feature flag. That’s worth looking into.

And of course, Cucumber. I also have a repository that is a starter report for working with Cucumber and Playwright. I’ll post it on the channel if anyone is interested to try it out. If you want to stick with the Gherkin and the BDD-style test. Otherwise, go with the Playwright test runner.

Andy Knight (moderator):

Tally, I want to plus one of both things you said there. First of all, everyone knows that I love BDD, I love Cucumber, I love Gherkin, so yes. But also, I want to go back to what everyone is harping about GitHub Actions. Y’all in the audience, if you haven’t tried GitHub Actions or used to yet, definitely check it out. It is very straightforward, low-to-no cost CI baked into your repo. I mean, like anytime I’ve done open-source projects or example projects, just putting a GitHub Actions to automatically run your tests from a trigger, oh, it’s beautiful. It can be a little intimidating at first because you’ve got that YAML structure you’ve got to learn. But I have found it very easy to pick up. I feel like personally – and speakers back me up on this – I never sat down and read a manual on how to write a GitHub Action. I just kind of looked at GitHub Actions and copied them and tweaked them, and it worked.

Tally Barak:

And then you went to the marketplace and looked for some.

Andy Knight (moderator):

Yes, yes, yes. You know, and so you can find things that are out there already. And I’ve done it not just with Cypress and Playwright, but literally like every single test framework out there that I’ve touched. You know, if it’s like a Selenium thing, if it’s a Playwright thing, a Cypress thing. Any kind of test unit integration, end-to-end, you name it. Like, it’s awesome. But anyway, I just wanted a plus one to both of those and now I’ll yield to the rest of the speakers’ favored integrations and why.

Steve Hernandez (Selenium):

Can I give a plug for LivingDoc? We use a lot of the SpecFlow stuff in our stack, as you saw, like the runner. There’s a great SpecFlow+ Runner report that gets generated and can show how each of the threads are performing in your test suite so you can look at performance. But one of the things just really simply is the Azure Log Analytics ingestion API. So right in your test pipeline, you can just throw up a PowerShell script or some console application and send that data that you would have in your CI tool over into something like log analytics. I suppose there’s an equivalent for Amazon. But then you can correlate, you know, your telemetry from your application along with test data and start to see where performance issues are introduced.

But I guess coming back to my plug for LivingDoc is, I feel in a lot of cases our automated tests are actually the only documentation for some parts of the application as far as how they’re supposed to work because you’ve captured the requirements in a test. So they’re kind of like living documentation and LivingDoc literally is that. And it’s something that you can share with the business and the developers to see what you have for coverage and it’s human readable. I think that’s really important.

Carter Capocaccia (Cypress):

Well, now with GPT3, everything is human readable.

Andy Knight (moderator):

So I wanted to ask about visual testing and specifically with Applitools. I saw and heard that some of y’all had used visual testing with Applitools in your projects. In line with talking about integrations that you love, how do y’all use Applitools if you use it, and what value do you find you get out of it? Go ahead.

Carter Capocaccia (Cypress):

Yes, I want to bring up kind of why Applitools is a different breed of product when it comes to visual testing, though. So you really have two flavors of visual testing. One is a bitmap comparison where you take a screenshot of an image and it takes it and basically breaks it down into computer readable code and it compares a known standard against the new version. If there’s a certain percentage with diversions between the old version and the new version, it flags it.You can kind of already tell, well, if all I know about is just basically one to zeros and comparing the results here, it’s not really all that intelligent. Because I could have a piece of the page, let’s say it’s a headline. The headline updates every five minutes. Well, I don’t have time to just continually update this baseline screenshot.

Well, what a tool like Applitools does is it actually takes your screenshots and puts them through computer vision. And with computer vision you can now provide a lot more instruction to what do you want to compare? What’s important to you? Do you want to ignore certain parts of the page? When I compare X to Y, do I really consider, let’s say, a carousel where the images are being delivered by a CMS and they’re being changed on a daily basis? Do I really care that yesterday it was a picture of let’s say somebody’s front porch and today it’s a picture of a tree? So there’s a lot more intelligence that’s built into a tool like Applitools than there is with standard bitmap comparison. I think it takes the right kind of customer for Applitools. And when they do have that right kind of customer, it’s a pretty huge game changer, considering that it allows you to make a lot more informed decisions about the health of the visual appearance of your application versus, like I said, a bitmap comparison.

Basically, if you want to compare the two, one is analog, the other is digital. The analog is the map comparison. The digital way is Applitools. If we’re being honest with ourselves, the Applitools way is probably the best and right way to do this. I just know that there are a lot of teams that are making smaller scale applications that can get away with assuming higher risk in using a tool like a bitmap comparison tool.

Jose Morales (WebdriverIO):

I would like to add that we have been using Applitools for six months or so, and it was a great addition to our framework. It was reducing a lot of code because we were coding a lot of assertions. So for example, if you go to the Domino’s website, you have the store locator, and then the store locator basically based on your zip code automatically populates the city region and the store closest to you. There is a lot of functionality there. And I remember that we were validating every single text fill, every single button, every single drop and menu for the content. In English and in Spanish and the different languages. So for us it was very helpful to use Applitools because with WebdriverIO, we can simulate the location from the customer and then move the store locator for that particular zip code and tell Applitools, hey, take a screenshot for this one.

And in one single shot, we are validating all the colors, buttons, text fields, content, and we were very careful to see what is worth it for this scenario, right? So in all the framework, what we are doing is okay, let’s test this particular functionality for this scenario. Let’s use WebdriverIO to move the customer until this particular checkpoint and then take a screenshot for that particular state in our application. And in one single shot, we can validate this in all the devices. In Chrome, Firefox, Edge. Pixel, iPhones. So with that particular action, and because we are reducing a lot of assertions, we are writing less code, it’s easy to maintain, it’s easy to extend, and our test cases are more accurate and smaller. And I think for us Applitools was a great addition and I fell in love with Applitools and currently we’re trying to incorporate the mobile validations, meaning iOS and Android as well.

Did anyone ever change their framework?

We heard popular tools mentioned like GitHub Actions and Cucumber, as well as some lesser known tools. A few of our panelists are also using Applitools Eyes in their projects to provide faster test coverage of their frontend. In the next article, we’ll cover the answers these experts gave when we posed a sticky question: “Did you ever consider changing your test framework?”

Let the Engineers Speak! Part 5: Audience Q&A

Let the Engineers Speak! Part 4: Changing Frameworks

Let the Engineers Speak! Part 3: Favorite Test Integrations

Let the Engineers Speak! Part 2: The Biggest Challenges

The post Let the Engineers Speak! Part 3: Favorite Test Integrations appeared first on Automated Visual Testing | Applitools.

Let the Engineers Speak! Part 2: The Biggest Challenges

Andrew Knight — Thu, 22 Dec 2022 14:20:00 +0000

In part 2 of our Cypress, Playwright, Selenium, or WebdriverIO? Let the Engineers Speak recap series, we will recap what our panelists shared as the biggest challenges they currently face in testing and automation. We’ll also cover some of the biggest challenges our webinar audience members have faced. Be sure to read our previous post, where we introduced the experts and their test automation projects.

The experts

I’m Andrew Knight – the Automation Panda – and I moderated this panel. Here were the panelists and the frameworks they represented:

Gleb Bahmutov (Cypress) – Senior Director of Engineering at Mercari US
Carter Capocaccia (Cypress) – Senior Engineering Manager – Quality Automation at Hilton
Tally Barak (Playwright) – Software Architect at YOOBIC
Steve Hernandez (Selenium) – Software Engineer in Test at Q2
Jose Morales (WebdriverIO) – Automation Architect at Domino’s

The discussion

Carter Capocaccia (Cypress):

So I think one of the biggest challenges that I personally faced was that a lot of the times – I think initially when you’re starting out with automation – the tech problems that you face are where all the focus gets put on. And people lose sight that there’s an entire component to process around this. Meaning that, well, once you’ve written, you know, your hundred tests, you’ve got your architecture stable, you’ve figured out how to seed your database and correct your APIs into all the stuff that you need to do to make your tests run. Well, what do you do with those results? How does it actually impact your app? And I think a lot of people lose sight of that.

Of course, with the advent of CI/CD pipelines, it becomes a little bit easier because now you can basically prevent merges or prevent releases based on the results of these tests. But I think that gets forgotten until we’re way down the line. So, you know, you’ve written all your tests and you come back to your dev teams and you say, “Hey, I’m gonna start failing your pipelines now, and you’re not gonna be able to merge, you’re not gonna be able to to deploy.” And then you immediately have resentment against this process, right? So I think maybe the marketing of why you’re doing what you’re doing, how it’s gonna impact the process, and what is the positive result that comes out of this is equally as important as tackling those technical hurdles.

Tally Barak (Playwright):

Yeah I think I will go somehow along those lines. Besides some technical difficulties that I already shared about the shadow DOM and trivial actions like dragging, again, some of the things that we encountered. The biggest challenge was creating a culture of testing in the organization, making sure that people take care of the test. Because of some flakiness, we didn’t want to prevent merges. I mean, [for] a long time, we built more and more into that, but at the beginning it was like, “People, this is your helper. This is something to help you write better code.” And it didn’t come very easily, I would say.

Poll results from over 60 webinar participants depicting their biggest challenges while testing

Gleb Bahmutov (Cypress):

Can I comment on this slide [of the] poll? I see selectors as one of the challenges and it’s [a] pretty big one, apparently, for a lot of people. I just want to say what we’ve done at Mercari US. So, sometimes [when] you’re writing an end-to-end test, you come to a part of a page and there [are] some dynamic classes, no good selector specific for testing. So what we do in our test [is] we select however we can – by text, by position, you name it. But we do add a to-do comment. So in our test repo, we have lots of to-do [comments] like “needs better test selector”.

And we have [an] automated script that actually counts it and puts it in a rhythm. So we know how many to-dos for selectors we have total. And one of our objectives as a team is to drive a number down. So we [are] adding selectors and it’s kind of a good small task to do, but having good selectors is a key. So don’t just kind of assume someone else will add it for you. Have it as one of your automated priorities.

Andy Knight (moderator):

That’s really cool to actually have it kind of scan through and kind of pinpoint. So it’s almost like you’re self-linting. Wow, that’s awesome. If I may ask another question off of this topic of the challenges specifically around selectors. What other challenges have y’all faced with selectors or innovative solutions to try to get on top of them? Like, does anybody use smart selectors or visual selectors or anything at all?

Carter Capocaccia (Cypress):

Can I share my really staunch opinion on this? I personally don’t allow merge requests that have XPath selectors in them. I disallow it. It is a forbidden practice at our organization. We actually start with – we have a selector hierarchy that we try and follow. And I believe it’s Testing Library. And know it’s a really generic name for everybody in this room, but genuinely, it’s called Testing Library. And they have a selector pattern basically as a hierarchical representation based. I think it says, you know, accessibility selectors, and then at the very bottom, it even goes down to classes and then IDs. But it starts with accessibility selectors, and then content and then, you know, some other pieces that you should go through. And nowhere in that list is XPath.

I know that a lot of folks feel like XPaths are necessary, but I think that you’ve gotta find a way around them. I’m sorry if that hurts you there, Pandy. But yeah, in my opinion, I think XPaths are sometimes used as a crutch. And in today’s world where you have components that are conditionally rendered or potentially even, like for situations that I’m working in, it depends on what part of the country you’re in or what part of the world you’re in. Your page may render it completely differently than another. So therefore I cannot rely on a DOM structure to not be shifting at a moment’s notice, or there may be a CMS tied into this so they can add a component to the page. It’s gonna completely change the DOM structure. So we have to do better than that and, you know, tests more and more nowadays are being kept right alongside the application code.

And I think that, speaking back to what I was saying about kind of changing the process around what you do. Well, if I’m a test engineer, if I really can’t get to a DOM element any other way, why can’t I go into that component and add a data test ID attribute? I should have that ability as a test engineer, just like a UI dev should have the ability to come into my test and modify that. So that’s I think a way you can kind of shift culture when you even think about something as simple as DOM selectors. But yeah, the long-winded answer to say, I personally think that if you want to have maintainable selectors, avoid XPaths.

Jose Morales (WebdriverIO):

I would like to share my opinion in two ways. One is the challenges that we have in Domino’s, and another one is the selectors.

The challenge is the maintenance of the test cases. And basically what we are following is a good process. And with a good process, I mean we have a team that is reviewing the executions and we do analysis about the results. We figured out if we need to fix the test cases, we need to change the test cases because the feature is now different – or what is the root cause of the problem, right? We are following good practices in terms of development, like code reviews, pull requests. We’re doing peer programming. If some automation engineer is facing some kind of challenging problem/blocker, we have working sessions they requested for those working sessions. And we work together, right, as a team to figure out what’s the problem, what [are] the possible solutions that we can do. And it’s very active.

And commenting about the selectors, we encourage to use the ID. And in Domino’s we have a custom ID is the UUID that all the web developers add that particular identifier for us to use. So there is a close communication between the developers and the automation engineers to work together as a team, how we can help to [better test] our product. That’s another thing that we normally do. And respect as much as possible the page-object design pattern. That is what we are doing in Domino’s.

Tally Barak (Playwright):

The slide that was showing actually three items that were shining there, which is flakiness, maintainability, and selectors. And I think they’re all related to some extent. Because if your selectors are too accurate, like XPath with all the DOM structure, then you have a maintainability problem and you might have flakiness. And I have to say, if you are talking about the tools, this is probably the one thing that Playwright has solved in just a way that makes writing tests really easy. I was hearing the other people here and they went like, “Oh, we already forgot about this problem after working with two and a half years.” We used to work with XPaths because that was the only thing that could pierce the shadow DOM.

Now we are working with plain selectors. We’re keeping them to the minimum and mostly the user-facing selectors. So things like text usually. Maybe something like a header component and then a text. And we will know how to address the very exact selector. And that reduces the maintenance because this is the last part that is likely to change. And because of the way Playwright is working with the auto waiting, it’ll just wait, we don’t need to put some time out. It’ll know to wait until a certain time out for the component to show up.

You can write very small and very concise selectors. And the one other thing is that they have the strict mode, which means that if it finds two elements that correspond to the same selector, it’ll break the test. And one of the problems that we’ve seen previously is that it would go with the first element. So it might click a different button or something else, and the test will continue and it’ll only fade like three steps or five steps ahead. And this is really hard to understand what is the problem. Playwright here will just tell you. Of course, you can change if you want it or not. It’s an option. But if you do that, you get a lot less flakiness in your test because the first time it’ll encounter the element, it’ll tell you, okay, “I have two here. You’re probably on the wrong path. Go ahead and fix your test.” So all these three things, the maintenance, the flakiness, the selectors is one of the, I think, the greatest advantages of Playwright in these terms.

Andy Knight (moderator):

Awesome. So I know we’ve spent a lot of time talking about selectors and DOM and modeling the page. What design patterns do y’all use for or within your test projects? I know, I think it was Jose mentioned like they use page objects and they stick to it very rigidly – or maybe rigidly is not the best word, but they stick with page objects. Are y’all using page objects or are y’all using Screenplay? I know Cypress has those action or command methods, I forget. What are y’all using to help make your interactions using those selectors better?

Steve Hernandez (Selenium):

Screenplay.

Gleb Bahmutov (Cypress):

We don’t try to make the test super dry, right? Eventually, if we see a duplication of code, we will write a custom command. For example, we have custom commands for selecting by data test id. So it’s really easy by data test ID or ARIA label, we use custom commands. We almost never write page objects. We prefer little reusable functions, right? We need to create a user. Here’s a function, creates a user for you.

There is one advantage, though, because I like writing Cypress plugins, we can abstract something that is really specific and just move it to [a] third party public open source plugin, but we’ll reuse in our project. So we’ve been doing that a lot. I just looked, we have more than 20 Cypress plugins with our test report, but 17 out of them is what I’ve written. So, we take that abstraction and we make it and just have it by itself so it becomes more powerful and reusable. But small functions and custom commands for us.

Tally Barak (Playwright):

We are working with Cucumber. So this is basically our structure. Each step has the automation behind the scene, and then obviously you can reuse because the step is reusable in any of your tests. And we do try to build them sometimes a bit more, you know, a sophisticated kind of code, so you can reuse them in different places.

Jose Morales (WebdriverIO):

[About] using other design patterns – another thing that really helped us. And I saw a very interesting question about how do you integrate a backend with a JavaScript frontend? And that’s a very interesting question because for us, we try to consume APIs as well. So for example, in Domino’s, we have an application that is an interface for configuration for our stores. So every manager in our stores can configure certain parameters for the store, right? Like the store phone number. So they can go to a specific website and then change the store’s number, for example. So if you think about the testing perspective, you can go to the UI and change a particular store number and then hit an endpoint to refresh the backend for that particular store configuration.

And then you go to another backend to hit the store profile and then get the store profile with a request and read the JSON back, right? So you don’t need to interact with the UI all the time, but you can also integrate with the backend as much as possible. So that’s a very clean strategy, because it helps you to test faster and test better and reduce the flakiness.

And another thing that I recommend is to use the plugins, right? So explore the plugins that you can get in the industry. For example, in the backend and if you are interested in measuring the performance, you can use Lighthouse with WebdriverIO. So you can – with the standards in the industry – measure what is the performance for your web application, right? So we don’t need to reinvent the wheel. And that’s my recommendation.

Andy Knight (moderator):

Awesome. Awesome. So I know, Steve, right off the bat, you had mentioned the Screenplay pattern. I wanted to come back to that. What have you found has been good about using Screenplay over something like raw calls or page objects or something else?

Steve Hernandez (Selenium):

Personally, I haven’t used a lot of raw calls. I’ve had the luxury of using Boa Constrictor. It kind of wraps the Selenium stuff in a warm hug, and it handles a lot of the timing issues that you might run into. Boa Constrictor has a lot of built-in – I hear you guys talk about plug-ins – we have a lot of built-in interactions, they’re called. So they can be questions that you’ll ask of an API endpoint. They can be questions that you’ll ask of the page in the form of, “I’m gonna scrape this column here that I use over and over to compare to something in the future”.

So we’ll have our Gherkin steps and essentially backing each of those step definitions are these different reusable components or questions or interactions. That works very well with the level of parallel parallelism we’re doing. You know, if you’re firing off 50 threads at a time or 80 or more, we need stuff to be fast, not step on each other’s toes. So we write the interactions in a way where they’re very atomic and they’re not going to affect something else that’s happening on another thread. As far as page objects, I haven’t used those a whole lot.

What about integrations?

We’ve heard some common themes from both our panelists and our webinar viewers around their test automation challenges – selectors, maintenance, and flakiness. Gaining a culture of testing that embraces and adheres to the process is another challenge our engineers have faced in their projects. Our panelists each touched a little bit on how they approach these challenges in this recap. In the next article, we’ll cover the panelists’ favorite integrations for their test automation projects.

Let the Engineers Speak! Part 5: Audience Q&A

Let the Engineers Speak! Part 4: Changing Frameworks

Let the Engineers Speak! Part 3: Favorite Test Integrations

Let the Engineers Speak! Part 2: The Biggest Challenges

The post Let the Engineers Speak! Part 2: The Biggest Challenges appeared first on Automated Visual Testing | Applitools.

Let the Engineers Speak! Part 1: Real-World Projects

Andrew Knight — Mon, 19 Dec 2022 16:27:39 +0000

Over the past two years, Applitools has hosted a series of code battles called Let the Code Speak. We pitted the most popular web testing frameworks against each other through rounds of code challenges, and we invited the audience to vote for which did them best. Now, to cap off the series, we hosted a panel of test automation experts to share their real-world experiences working with these web test frameworks. We called it Cypress, Playwright, Selenium, or WebdriverIO? Let the Engineers Speak!

Here were our panelists and the frameworks they represented:

Gleb Bahmutov (Cypress) – Senior Director of Engineering at Mercari US
Carter Capocaccia (Cypress) – Senior Engineering Manager – Quality Automation at Hilton
Tally Barak (Playwright) – Software Architect at YOOBIC
Steve Hernandez (Selenium) – Software Engineer in Test at Q2
Jose Morales (WebdriverIO) – Automation Architect at Domino’s

I’m Andrew Knight – the Automation Panda – and I moderated this panel.

In this blog series, we will recap what these experts had to say about the biggest challenges they face, their favorite integrations, and if they have ever considered changing their test framework. In this article, we’ll kick off the series by introducing each speaker with an overview of their projects in their own words.

Gleb’s Cypress Project at Mercari US

Gleb:

I’m in charge of the web automation group; we test our web application. Mercari US is an online marketplace where you can list an item, buy items, ship [items], and so on. I joined about a year and a half [ago], and my goal was to make it work so it doesn’t break.

There were previous attempts to automate web application testing that were not very successful, and they hired me specifically because I brought Cypress experience. Before that, I worked with Cypress for years, so I’m kind of partial. So you can kind of guess which framework I chose to implement end-to-end tests.

We currently run about 700 end-to-end tests every day multiple times per feature with each pull request. On average, our flake – you know, how many tests kind of pass or fail per test run – is probably half a percent, maybe below that. So we actually invested a lot of time in making sure it’s all green and working correctly. So that’s my story.

Carter’s Cypress Project at Hilton

Carter:

Nice to meet you all today and thank you for the opportunity to speak with everybody. So I need to lead off with a little bit of a disclaimer here. Hilton does not necessarily endorse or use any of the products you may speak of today, and all the opinions that I’m going to talk about are my own.

So Hilton is a large corporation. We operate 7,000 properties in 122 countries. So when we talk about some of the concerns that I face as a quality automation engineer, we’re talking about accessibility, localization, analytics, [and] a variety of application architectures. We might have some that are server-side rendered. We might have some that are CMS-driven pages and even multiple types of CMS. We also have, you know, things that are like that filter out bot traffic. We may use mock data, live data. We have availability engines, pricing engines. So things get pretty complex and therefore our strategy has to follow this complexity in a way that makes things simple for us to interpret results and act upon them quickly.

To give you guys an idea of the scale here, we typically are much like Gleb in this regard, where every single one of our changes runs all of our tests. So we are constantly executing tests all of the time. I think this year – I was just running these reports the day to figure out for, you know, interview review stuff – I think I pulled up that we had so far in this year run about 1.2 million tests on our applications. That’s quite a large quantity. So, you know, if anything, that speaks to our architecture – [it] has some stability built into it – the fact that we were even able to execute that quantity of tests.

But yeah, I’m a Cypress user. I’ve been a Cypress user now for about three years. Much like Gleb, I got faced with the challenge of, “Well, this only runs in Chrome. It’s a young company, young team.” And yeah, I think, you know, we stuck with it, and obviously today Cypress has grown with us and we’re really, really happy to still be using the tool. We were a Webdriver shop before, so I’m really interested to see what Jose is doing. And there probably is still some Webdriver code hanging around here at Hilton. But yeah, I’m really excited to be here today, so thank you for the opportunity.

Tally’s Playwright Project at YOOBIC

Tally:

I work at YOOBIC. YOOBIC is a digital platform for frontline employees – that’s a way to say people like sellers in shops. And they use our application in their daily job – for the tasks they need to do, for their training, for their communication needs, and so on.

For this application to be useful, we have a very rich UI that we are using. And for that, we have decided to use web components. We’re using extensive JS and building web components. This is important because when we tried to automate the testing, this was one of the biggest barriers that we had to cross.

We used to work with WebdriverIO, which was good to some extent. But the thing that really was problematic for us was the shadow DOM support – and shadow root. So I got into the habit for once every fortnight or monthly to go into the web and just Google “end-to-end testing shadow DOM support” and voila! In one of these Googlings, it actually worked out and I stumbled upon Playwright – and it said, “Yes, we have Shadow DOM support.” So I went like, “Okay, I have to try that.” And I just looked at the documentation. It was like shadow DOM was no biggie. It was just like built/baked in naturally. I started testing it, and it worked like magic.

That was almost two and a half years ago. Playwright had a lot less than what it has today. It was still on version 1.2 or something. At the time, it was only the browser automation side; it didn’t have the test runner. And our tests were already written in Cucumber, and Cucumber is a BDD-style kind of testing. So this is the way we actually describe our functional test. So user selects mission, and then user starts, and so on – this is the test. We already had that from WebdriverIO, and basically we just changed the automation underneath to work with Playwright instead of Selenium WebdriverIO.

And this is still how we work. We are using Cucumber for running those tests. We use Playwright as the browser automation. We have our tested app and we are testing against server and the database. We have about 200 scenarios. We run them on every PR. It takes about 20 or 25 minutes with sharding, but this is because they are really long tests.

It also solved a lot of other problems that we had previously, like the fact that you can open multiple browsers. Previously what we were doing, if you wanted to test multiple users, we would log in, log out, and log in with the other user. And here, all of a sudden, we could just spawn multiple windows and do that.

And one [additional] thing that I want to mention besides the functional test, this end-to-end test, we also run component tests. This is different; we are using the Playwright Test Runner, in fact. This is because we work with Angular, and Playwright component testing is not supported with Angular yet. We are using Storybook, and we test with Playwright on top of Storybook. So yeah, that’s my story.

Steve’s Selenium Project at Q2

Steve:

I actually started as a full stack developer at Precision Lender. We’re part of Q2, which is a big FinTech company. If you had priced any skyscrapers or went to get a loan for a skyscraper, there’s a very good chance that our software was used recently – I’m sure you’re all doing that.

Working on the front end, it’s a JavaScript-based application using Knockout but increasingly Vue. I worked over in that for about two and a half years at the company, and I saw the cool stuff that Andy and our other Andy were doing. And I was excited about it. I wanted to move over [and] see what it was all about. And I believe strongly in the power of test to not only improve the quality of the application, but also just the software development lifecycle and the whole experience for everyone and the developers.

Q2 test automation architecture diagram from the Let the Engineers Speak webinar

So what you guys have in front of you is a typical CI pipeline that we use. We use Git and we use TeamCity. And on a typical day, we run about 20,000 tests. We have about 6,000 unique tests and upon one commit to our main developed branch, we kick off about 1,500 tests and it takes about 20 to 25 minutes, depending on errors, test failures. On that check-in to Team City, we use a stack with SpecFlow and SpecFlow+ Runner kicks off our tests.

As I said before, we’re using Boa Constrictor underneath with the Screenplay Pattern. It acts as our actor, which is controlling the Selenium Webdriver. And then the Selenium Webdriver farms out the test work in parallel to the Selenium Grid and our little nodes; it’s an AKS cluster run through all of our tests and spit out a whole bunch of great documentation. One of my favorites being the SpecFlow+ Runner report and other good logging things like Azure Log Analytics.

Jose’s WebdriverIO Project at Domino’s

Jose:

Thank you from all [of us at] Domino’s. Well, Domino’s is a big company, right? We have 4 billion dollars in sales every single year. And – only in the Super Bowl – we sell 2 million pizzas just in the United States. So we have a big product and our operations are very critical. So I decided to use WebdriverIO for our product. And this is the architecture that we created in Domino’s.

Domino’s test automation architecture diagram from the Let the Engineers Speak webinar

In Domino’s, we have different environments. We have QA, we have pre-production, and we have production. And for selecting different kinds of environments, we have different kinds of configuration. And in that configuration, Jenkins reads the configuration for that particular environment, we get the code from a stash, and then we are using virtual machines – it could be inside of the Domino’s network or it could be in Source Labs. Then we have a kind of report. And one thing that I really love about WebdriverIO is the fact that it’s open source, and it has huge libraries.

So in Domino’s, we’re using WebdriverIO in three projects – Domino’s website, in the Domino’s re-architecture, and in the Domino’s call center application. [In fact,] we are using WebdriverIO with two different kinds of reports, and we’re using [it] with two different kinds of BDD frameworks. One is, of course, Cucumber; that is the most used in the industry. And another one is with Mocha. And in both, we have very successful results. We have close to 200 test scenarios right now. And we are executing every single week. And we are generating reports for those specific scenarios. We have a 90% success pass rate in all scenarios.

And we’re using Applitools as an integration for discovering issues – visual issues and visual AI validations. And basically in all those scenarios, we’re validating the results in all those browsers. We’re normally using Chrome, Safari, Firefox, Edge, and of course, we’re using mobile as well, in the flavors of Android and iOS.

And the cool thing is that it really depends about what is the usage for customers, right? Most of the customers that we have are using Chrome, and we are seeing what is our market, what our customers are using, and we are validating our products in those particular customers.

So I am really happy with WebdriverIO, because I was able to use it in other problems that we were facing. For example, in Malaysia we have some stores as well, and in the previous year we had a problem with performance in Malaysia. So I was able to integrate a plugin – Lighthouse from Google – in order to measure the performance for that particular market in Malaysia using WebdriverIO. So far for us [at] Domino’s, WebdriverIO is doing great. And I am really happy with the solution and the results and the framework that we selected. And that’s everything from my end. Thank you so much.

What are their biggest challenges?

So we’ve gotten an overview of the projects these test experts are using and a little bit of their histories with their test framework of choice. This will help set a foundation to get to know our engineers and understand where they’re coming from as we continue our Cypress, Playwright, Selenium, or WebdriverIO? Let The Engineers Speak series. In the next article, we’ll cover the biggest challenges these experts face in their current test automation projects.

Let the Engineers Speak! Part 5: Audience Q&A

Let the Engineers Speak! Part 4: Changing Frameworks

Let the Engineers Speak! Part 3: Favorite Test Integrations

Let the Engineers Speak! Part 2: The Biggest Challenges