The post Automatically Run Visual Tests on Every Netlify Deploy appeared first on Automated Visual Testing | Applitools.
]]>
Running tests shouldn’t be an afterthought, but it can seem challenging to dive in with the huge variety of frameworks and simply not knowing what to test. Instead, we can use Netlify Build Plugins to instantly add automated visual testing to every deployment in just a few clicks with the Applitools Visual Diff Plugin.
What is a Netlify Build Plugin?
Netlify is a hosting and deployment platform that automates taking a web app and publishing it to the web. This also includes the ability to deploy serverless functions, allowing developers to closely develop APIs and backend logic right next to their apps.
As part of that process, Netlify gives developers the ability to install Build Plugins, which helps extend the default deployment allowing additional tools and services to tap.
This makes for a perfect opportunity for things like code analysis or custom build caching for those who don’t want to maintain it themselves, but it’s also an opportunity for adding testing, where we can start using Applitools to run visual tests without having to have any other type of infrastructure or frameworks configured.
How does Applitools work with Netlify Build Plugins?
With Build Plugins, Netlify provides a few different hook locations, meaning, throughout the build and deploy process, there are different steps at which point a Build Plugin can run code.
One of those steps is called Post Build, where as soon as Netlify finishes building the application, we can determine all of the pages that were built and run a visual test with Applitools Eyes.
This provides instant, broad coverage for web apps, making sure that each time we deploy the site, we’re not creating any new regressions or visual bugs.
How to get started with the Applitools Build Plugin?
If you’re already up and running with Netlify, the only thing you need to get started is a free Applitools account. Otherwise, this plugin only works for Netlify-deployed applications.
With your account, you’ll want to be able to locate your Applitools API key. You can find your API key inside of your Applitools dashboard under the account dropdown and by selecting My API Key.
Once you’re ready to go, there are two ways to install the Visual Diff plugin: using the Netlify dashboard or using the Netlify configuration file in your project.
Note: to use advanced settings such as custom browser configurations and ignore regions, you’ll need to use Option 2, using the Netlify configuration file in your project.
Option 1: Adding the Applitools Visual Diff Plugin with the Netlify UI
Setting up your API key
To start, locate to your site inside of Netlify and navigate to the Site Settings section.
Once inside, select Build & deploy in the sidebar and scroll down to Environment.
Here, we want to click Edit variables, where we’re going to add a new variable.
For the name, add APPLITOOLS_API_KEY.
For the value, add your unique Applitools API key.
Adding the Applitools Visual Diff Build Plugin
Next, navigate to the Plugins section of the Netlify dashboard, where it will then give you an option to go to the Netlify plugins directory.
Here, you can either search for “Applitools” or scroll down to the bottom where the name of the plugin will be “Visual diff (Applitools)”.
Click the Install button, where it will then ask you to select which site you want to add it to. Find and select your site then confirm installation.
Running a visual test through Netlify
Applitools has the ability to run visual tests by hooking into the Netlify build and deploy process, which means, we need to trigger a new deploy.
Inside of the Netlify dashboard and navigate to the Deploys section.
On the right, above the list of recent deployments, where you’ll see a button that says Trigger deploy. Click Trigger deploy, then select Deploy site from the dropdown.
This will now kick off a new deployment with Netlify. You can even view the logs if you select the new deployment.
Viewing Applitools Visual Testing results
Once the Netlify deployment as finished, Applitools will have run through your site visually testing each page.
You can now head over to the Applitools dashboard, where you’ll now see a new test with your project’s name!
Every time you run your test after, Applitools will compare the active state of your project to this baseline and make sure everything is working as expected!
Option 2: Using a netlify.toml to add the Applitools Visual Diff Plugin to a project
Setting up your API key
Even though we’re going to be setting up the plugin with our Netlify configuration file, we’ll still need to add our API key via the UI.
To start, locate to your site inside of Netlify and navigate to the Site Settings section.
Once inside, select Build & deploy in the sidebar and scroll down to Environment.
Here, we want to click Edit variables, where we’re going to add a new variable.
For the name, add APPLITOOLS_API_KEY.
For the value, add your unique Applitools API key.
Adding the Applitools Visual Diff Build Plugin
If you don’t already have one, create a netlify.toml file inside of the root of your Netlify project.
Next, add the following:
[[plugins]]
package = "netlify-plugin-visual-diff"This will tell Netlify that you want to use the Visual Diff plugin when deploying your site.
Running a visual test through Netlify
With your new configuration, commit the new file changes and push them out to your Git repository that’s connected to Netlify.
Unless disabled, this will automatically kick off a new Netlify deploy for your site which will run the visual tests during the build process!
Note: if you have automatic deployments disabled, try going to the Deploys section and triggering a new build.
Viewing Applitools Visual Testing results
Once the Netlify deployment as finished, Applitools will have run through your site visually testing each page.
You can now head over to the Applitools dashboard, where you’ll now see a new test with your project’s name!
Every time you run your test after, Applitools will compare the active state of your project to this baseline and make sure everything is working as expected!
Optional: Advanced Configuration
If using the netlify.toml to manage the plugin, you additionally have the option of passing in configurations via Netlify build plugin inputs.
Head over to the [Applitools Visual Diff plugin GitHub](Viewing Applitools Visual Testing results Once the Netlify deployment as finished, Applitools will have run through your site visually testing each page. You can now head over to the Applitools dashboard, where you’ll now see a new test with your project’s name! Every time you run your test after, Applitools will compare the active state of your project to this baseline and make sure everything is working as expected! ) to learn more!
Learn more about Applitools
Getting started with Netlify and the Applitools Build Plugin is just one of the many ways you can provide broad visual testing coverage on your project.
Check out the Applitools tutorials to find your favorite framework or get in touch to schedule a demo with one of our engineers!
The post Automatically Run Visual Tests on Every Netlify Deploy appeared first on Automated Visual Testing | Applitools.
]]>The post Using GitHub Copilot to Automate Tests appeared first on Automated Visual Testing | Applitools.
]]>
Code completion is nothing new. Tools like IntelliSense have allowed developers to become more productive by attempting to automatically complete the name of a function or statement they’re in the middle of writing, but the tools available only have a certain level of actual “intelligence” available.
With GitHub’s limited release of Copilot, they’re taking advantage of the powers of AI to boost developer productivity by not only trying to complete a function’s name, but by trying to suggest the entire function itself!
What is GitHub Copilot?
GitHub Copilot is a new tool from GitHub that provides realtime code suggestions when working inside of VS Code as an extension.
It’s powered by OpenAI, trained on billions of lines of public code, courtesy of projects hosted on GitHub itself, giving it the ability to have a wide variety of authors and languages to base those suggestions off of.
How does it work?
Copilot will typically give suggestions as you type in any scenario, but the two compelling examples that GitHub provides is being able to create a function based off of a comment or based off of the name of the function.
For instance, if I were to write:
// returns a sorted array of objects based on dateCopilot will get started with a new function, then once accepted, continue to write that function.
Alternatively, I can supply the function name like:
function sortArrayByKeyWhere Copilot could then write that entire function for me!
If I don’t like the first answer it gives, I also have the option to look through alternatives, including pulling up all the answers Copilot thing could be helpful in this context.
What other kinds of things can it do?
There aren’t really any hard limitations beyond the basic gist of the tool being a way to suggest code. That just means you’ll be able to get a suggestion for finishing a function, but not necessarily for scaffolding a project with new files and directories.
While it can suggest really anything like an entire React component, it seems to really excel at utility functions that have a clear goal such as custom functions to sort an array like above or filtering data. Building components can be pretty specific and not include dependencies, where those functions often only require a specific input and output.
Copilot can even read context from the active file. If you’re trying to sort or filter data based on what exists in that file, it will try to use that context in its suggestion.
Can it write tests?
Writing functions is one thing, but can Copliot produce real, meaningful tests? Well, it depends.
Context is an important factor with tests. If you’re writing a unit test with Jest to harden business logic or a Cypress test to walk through your UI, that information is typically scattered in other files or not even available until it renders in a browser. Copilot can write a test based on similar situations, but it wouldn’t “just work”.
That said, when using testing frameworks like Applitools, you can get visual testing coverage with code that doesn’t need to be super specific to your application, only requiring it to load in the browser, where Applitools steps in and does the heavy lifting.
Part of Copilot’s wide net of training data includes a variety of Applitools examples, meaning we can easily write our new Eyes checks right in our existing testing frameworks.
Is Copilot the future of development?
The release of Copilot and tools like it bring a new era in what the software we’re using can do for us in day-to-day development. By leveraging AI and machine learning, we’re able to train computers to do the hard lifting and mundane tasks while we focus on solving the real problems, not how to fight with our text editor.
For a lot of developers, getting this information is a typical Google search with a look at a Stack Overflow answer. Even if only using it for common functions, we’re saving time and mental resources not having to look those things up ourselves.
But Copilot is still just another tool in our belt. It shows us what’s possible, but like all other tools, it’s up to the developer to ultimately take that knowledge and use it to its potential.
The post Using GitHub Copilot to Automate Tests appeared first on Automated Visual Testing | Applitools.
]]>The post Adding Vision to Cypress Component Test Runner with Applitools Eyes appeared first on Automated Visual Testing | Applitools.
]]>
Modern applications typically revolve around the idea of smaller components, where you may have pieces like a button, navigation, or an interactive widget that all fit together like puzzle pieces to make up a larger app experience.
When trying to test these components, we’re usually stuck having to load the entire application just for that one piece or using synthetic rendering. This allows us to simulate how interacting with a component would work, but it’s not necessarily the same as the real thing.
Instead, we can use Cypress’ newly released Component Test Runner which looks to help make testing components more reliable and easier.
What is the Cypress Component Test Runner?
The Cypress Component Test Runner is a new solution from Cypress that allows you run your component tests in the browser just like someone visiting your app would actually use it.
These tests would sit next to your typical end-to-end tests, where the goal is to create tests that are focused on each individual component rather than the full application.
Because component tests don’t require page routing or loading the rest of the application, these tests can run much faster and with less overhead.
Out of the box, Cypress supports popular component-oriented libraries like React and Vue along with bundlers like Webpack.
Adding vision to component tests with Applitools
Cypress does a great job being able to leverage the browser so that our tests are using real-world environments, but any assertion that you write is still going to be based on the code.
This starts to become a problem when we’re only able to test how our application is functionally working and not necessarily how someone can actually use it.
That’s where Applitools steps in! With the Cypress Eyes SDK, we’re able to provide comprehensive visual testing for each of our components..
How Applitools works with the Cypress Component Test Runner
We can use the same Applitools Cypress Eyes SDK that you can use to add visual testing to your Cypress end-to-end tests to add visual testing to your Cypress component tests.
Because Cypress uses the same APIs and plugin architecture, once the Eyes SDK is installed, running visual testing is as simple as opening your Eyes and capturing the component.
Inside of a Cypress component test, you can use the mount function to load your component:
mount(<MyComponent />);As soon as it’s mounted, to run Applitools Eyes, include:
cy.eyesOpen({ appName: 'My App', testName: 'MyComponent' });
cy.eyesCheckWindow();
cy.eyesClose();And Eyes will capture a DOM snapshot of your rendered component and send it to the Applitools cloud where it will check for visual differences.
Learn more about Applitools and the Cypress Component Test Runner
If you want to get started by trying out a working example, head over to this GitHub repository showing how to use Applitools and Cypress with Create React App.
https://github.com/colbyfayock/my-component-tests
To check out the launch announcement, make sure to watch the opening keynote from Front-End Test Fest.
Otherwise, learn more about how to use Applitools with Cypress with the Eyes Cypress SDK.
Get Started Today
Want to try Applitools Eyes for yourself? You can get started today at the link below – the account is free forever.
The post Adding Vision to Cypress Component Test Runner with Applitools Eyes appeared first on Automated Visual Testing | Applitools.
]]>The post How to Maintain High Quality Design Systems with Storybook by Leveraging Visual AI appeared first on Automated Visual Testing | Applitools.
]]>
Component libraries and design systems are important development and design tools that allow teams to focus on building consistent and high quality experiences. Storybook is a tool that helps us with those experiences, but as your library and system grows, it becomes more difficult to maintain that level of high quality.
With the Applitools Eyes Storybook SDK, we’re able to provide visual testing coverage for all stories in your Storybook. The cool thing is this works for whatever UI framework you choose! Whether you’re using Storybook with React, Vue, Angular, Svelte, or even React Native, Applitools is able to support any framework Storybook supports.
Let’s dig in and find out what exactly Storybook is, where it helps, and how Applitools can easily provide coverage for any number of stories that a Storybook library supports.
What is Storybook?
Storybook is a JavaScript-based tool that helps teams build component libraries, design systems, and beyond.
It does this by providing a library-like interface, that shows different components and pages in isolation allowing developers, designers, testers, and users of the library to consume and work in each “story” in a focused environment.
Each Storybook story is composed of a component or set of components that represent one piece of what someone would interact with or visualize in an application.
Testing and validating UI at the component level
While most stories don’t show an entire page built with these components, each of them are fit together in various ways to build the actual interfaces that are used by visitors of that app.
This gives the development team or users of the library an easy and flexible way to make sure each component works as expected from the start, where they can then have confidence to use it however they’d like in the application.
But the tricky part is making sure we’re providing proper coverage of each of the components in a way that will help us build that confidence along with finding a scalable solution as the library grows.
Solutions exist to provide code-based testing, where we might validate the string-based output of a component or test the components virtually mounted, but they’re not actually capturing what the users of the components are actually seeing.
Visual testing helps us solve that. We can capture what the user is seeing right in the browser, and use that to make sure our components and stories are working exactly like they should, giving us that needed confidence.
AI-Powered Visual Testing and Storybook
With the Applitools Eyes Storybook SDK, we’re able to easily provide coverage for any number of stories that our Storybook library supports.
After installing @applitools/eyes-storybook, all you need to do is simply run the following command:
npx eyes-storybook
Once run, Applitools will find all stories and start to capture each story much like other SDKs available for Eyes.
Because Storybook is ultimately web-based, where it renders any UI framework it supports right in a browser, the Eyes SDK is able to capture a DOM snapshot of that rendered output.
That means, the Eyes Storybook SDK can support any framework that Storybook can support!
The SDK uses Puppeteer, similar to the Eyes Puppeteer SDK, where each story is loaded in a new Puppeteer tab.
It then collects a DOM snapshot which is used to render each page cross-browser in the Applitools cloud, where you’ll then get the same AI-powered Visual Testing coverage that you would with any other Applitools integration.
This helps so that we can visually identify the issues with our application, based on what people are actually seeing. Saving us time when trying to solve where the issue occurred or if it’s a false positive in the first place.
Getting started with Applitools Eyes and Storybook
If you want to get started with integrating Applitools Eyes with your Storybook library, we have a variety of resources including a walkthrough for adding Applitools Eyes to Storybook along with dedicated tutorials for React, React using CSF, Vue, and Angular.
You can also learn more about the importance of design systems with my free webinar Bringing Quality Design Systems to Life with Storybook & Applitools.
The post How to Maintain High Quality Design Systems with Storybook by Leveraging Visual AI appeared first on Automated Visual Testing | Applitools.
]]>The post Migrating End-to-End Protractor Tests for Angular Projects appeared first on Automated Visual Testing | Applitools.
]]>
Protractor, the Angular end-to-end testing tool, is officially being sunset. With development ending at the end of 2022, we need to assess what our options are for further investing in Protractor as the tool that maintains our end-to-end testing suite. Here, we’ll take a look what our top Protractor alternatives are moving forward and how we can stay confident in our tests.
Why is Protractor being deprecated?
The Protractor GitHub issue that announces the deprecation plans goes into a lot of detail about why the decision was made and how it was not made lightly, but the theme of the decision is that the JavaScript ecosystem was in a completely different state back in 2013 when Protractor was created.
WebDriver APIs weren’t standard and this was before popular end-to-end tools like Cypress and Playwright existed.
Fast forward to 2021, where developers have a wide variety of options for their end-to-end tests, giving teams a way to make sure their needs are met for their individual use cases.
The Protractor team inside Google ultimately made a decision weighing the cost of updating Protractor and what that migration effort would look like for users, where it made more sense to help guide those users to other solutions.
What are our options moving forward for Protractor alternatives?
Luckily there are a lot of great testing tools with ample documentation that should make the transition smooth.
To get a sense of what tools we can use, we’ll look at some of the popular options we can use along with a basic example of a test with that particular tool.
We’ll start with this example of a test inside of Protractor:
Inside of this test, we’re loading up The Kitchen’s Select ingredient where we select an item (garlic) from that dropdown and make sure it successfully has that value.
So now, let’s dive into what other frameworks we have available as alternatives to Protractor.
Cypress
Cypress is a tool that has become very popular in the JavaScript world. Cypress has an API much like Playwright and Puppeteer which we’ll see shortly, but it goes an extra step to try to make the experience of writing and running tests better.
For one, unlike the others, Cypress ships with an assertion library out of the box (Chai). While it’s a small thing, it’s one less piece to configure.
Cypress additionally has a testing UI that pops up when running tests locally, giving flexibility to run only the tests you want and see the results along with debugging options in realtime.
The first thing we’ll notice is the code is short and concise.
Here we visit the page, which includes both launching a browser and navigating to the page. Then, in one line, we can get our Select dropdown, select our option, and check that it has the value we need.
While some of the other tools might have more flexibility, the Cypress API provides a slick way to easily automate your browser interactions.
Playwright
Playwright is an end-to-end testing tool from the team at Microsoft. It allows automation of modern web apps across browsers like Chrome, Firefox, and Webkit.
Playwright is actually quite similar to Puppeteer, which we’ll cover next, where some of the same core members who built and maintained Puppeteer actually moved to Microsoft from Google to create Playwright.
Where Puppeteer was a Chromium-based tool only, Playwright steps in and provides support for all modern browsers including Google Chrome and Microsoft Edge by using Chromium, Apple Safari by using WebKit, and Mozilla Firefox.
Using Playwright, we can choose what browser we’d like to run our tests on (or multiple browsers) and issue some straightforward commands for Playwright to follow.
In the above, we’re using Chromium, where we’re able to launch a new browser instance along with a new page, navigate to the Select ingredient, select our option, and check it’s value.
Note, similar to others on this list, you’ll need an additional library like Jest to write assertions.
Tip: if you want to learn more about Playwright, check out our article Playing with Playwright!
Puppeteer
Puppeteer is a similar tool to Playwright brought to you by the Google team. It followed what was an initial release of headless Chrome, where with Puppeteer, you can automate tasks right inside of the Chrome browser using native Chrome features. This helps to prevent flakiness some other frameworks tend to have.
As we dive into the code, you’ll notice that Puppeteer looks really similar to the Playwright example, minus a few small syntax changes. This is due to the history we talked about above, where Playwright was built by the team who used to work on Puppeteer.
In the above, just like Playwright, we launch our browser and navigate to our page, select an option from the dropdown, and make sure it’s what we expect it to be.
While production support is only available for Chrome, other browser support appears to be on its way, starting with Firefox.
Similar to Playwright, we need a library like Jest to handle our assertions.
Selenium WebDriver
Selenium WebDriver is a tool that’s been around for quite a long time, long before the newer Playwright, Puppeteer, and Cypress.
Selenium uses the WebDriver protocol to interface with the browser which uses a server to interact with the browser like ChromeDriver or the geckodriver for Firefox.
In our test, we need to create a new driver where we configure it as a new instance of Chrome. with our driver, we can navigate to our page just like any other tool, where we then use the API to find our elements, interact with our elements, and make sure that it’s working as expected.
We can also see in the above that we pull in assert which we use to write the assertion and verify it’s working as expected.
TestCafe
TestCafe is a framework that touts its ability to run tests out-of-the-box without any need for WebDriver. It also has the capability of running on all modern browsers.
While TestCafe is still based off of JavaScript, its syntax looks a bit different compared to the other ones we’re reviewing.
In the above, we first define a fixture which will be the page that we want to visit in our test. We next define our selectors before we even get to the test, which in this instance, we want to define our dropdown as well as the option that element which will be a child of the dropdown.
With our selections, we click our way through, selecting our element by its label instead of its value, then make sure its working as expected.
We can see that we get the assertion here out of the box without needing another dependency.
Webdriver.io
Webdrverio.io is a tool that can run on both the WebDriver and Chrome DevTools protocol to automate browser testing. As standards have changed, their support of running both gives developers the flexibility of using whichever option works for their use case.
It’s also capable of native mobile testing through an emulator, simulator, or on device, as well as native desktop applications written with tools like Electron.js.
While Webdriver.io primarily runs on WebDriver, you’ll notice the syntax is a bit more similar to some of the newer frameworks we’ve discussed.
Here, we’re loading up a page by URL just like any of our other tests and similar to Cypress, we’re able to do it in one line. Then, we find our Select element using a jQuery-like syntax, where we can both select our value and get the value, to verify it’s correct.
In this example, we’re pulling in Jasmine as our way to add assertions.
End-to-End Visual Testing with Applitools
The good news, is whatever testing tool you choose as your Protractor alternative, Applitools already has you covered.
Applitools supports a wide variety of SDKs that interface directly with your testing tool of choice, making integration super easy.
To find examples of using Applitools with all of the tools above along with many others, check out the Applitools tutorials page.
The post Migrating End-to-End Protractor Tests for Angular Projects appeared first on Automated Visual Testing | Applitools.
]]>The post Testing Browser Alerts, Confirmations, and Prompts with Cypress appeared first on Automated Visual Testing | Applitools.
]]>
A major component of building for the web is providing our customers feedback as they’re navigating and using our apps. The browser provides native ways to do that including alerts, confirmations, and prompts, but how can we use automation tools like Cypress to test those experiences?
What are alerts, confirmations, and prompts?
While developers tend to create custom solutions for providing a notice to someone visiting their website, all browsers actually come with a few methods right out of the box to handle this for you including alerts, confirmations, and prompts.
They all provide similar functionality, but vary slightly in the feedback requested from the site visitor.
Alerts: triggers a message dialogue with a single action allowing the user to accept and close the dialogue.
alert('This is an alert!');Confirmation: triggers a message dialogue but provides two actions allowing the user to accept or cancel the dialogue, both closing after an action has been taken.
confirm('Would you like to confirm?');Prompt: triggers basically the same thing as a confirmation, except there’s an additional text input, for the visitor to return a message
prompt('Please enter your confirmation.');All of the options are somewhat similar, but provide a different set of actions which can be useful for different situations.
Challenges with testing some native browser features
The issue with trying to test these native browser features is they’re not available in the DOM (Document Object Model). Cypress and other testing frameworks take advantage of JavaScript’s ability to query selectors.
For instance, if I wanted to select a button on the page with Cypress, I might run:
cy.get('#my-button')While the Cypress get method is a bit more complex, we’re essentially looking for that element by a selector, similar to:
document.querySelector('#my-button');When dealing with the alerts, confirmations, and prompts, we don’t have a way to select these elements.
Alerts, confirmations, and prompts in Cypress
To get around this limitation, we can instead listen to the events that these dialogues emit to make sure our notifications are working as expected.
For example, if we wanted to listen for the alert dialogue, we could add an event listener in Cypress like:
cy.on('window:alert', callback);While this method is still somewhat limited and it varies a bit between the dialogue types, it should give us enough functionality to regain confidence in our tests by allowing us to confirm the text as well as confirming the event actually occurred.
Let’s dig into the code and see how each of these would look in practice.
Testing alerts in the browser with Cypress
Similar to what we saw above, we can test our alerts using code similar to the following:
After first triggering the alert, we’ll listen to the window:alert event. When it’s triggered, Cypress will pass the argument text containing the alert’s message to the callback function, which we can then write an assertion against.
Testing confirmation in the browser with Cypress
Testing a confirmation dialogue will be similar to our alert:
Like the alert, we’re testing to make sure our message (text) includes exactly what we want.
The tricky part however, is we don’t have an explicit way to test the response as to whether the person hit “Ok” or “Cancel”. So we have to assert against something in the UI, where in this example, we have an element that contains the message.
Speaking of having two options, by default, Cypress will automatically hit “Ok” in our confirmation. If we wanted to instead test a use case of someone clicking “Cancel”, we can instead return false in our event callback:
And similarly, if you want to test that someone cancelled, you’ll need to have a way to indicate that in the UI.
Testing prompts in the browser with Cypress
Prompts are where things get slightly different. Let’s dive right in:
You’ll notice that we’re not testing this using a Cypress event like we did before. That’s because Cypress doesn’t include a window:prompt event similar to our alerts and confirmations.
Instead, we’re going to first access the window using cy.window(), where we’ll then “stub” that prompt event, along with the message that we want Cypress to return to the prompt dialogue.
Once stubbed, just like the other two, we can click our button and make sure that our UI includes an indication that the answer worked.
Now like our confirmation, the prompt dialogue has the ability for someone to click “Cancel”.
To test out the cancel path, we can use the Cypress callsFake method and return null (it has to be null), which will then tell Cypress that we want to cancel that prompt when it gets triggered.
What to expect in Cypress
After implementing these tests, you might be wondering why you’re not actually seeing these dialogues inside of Cypress itself.
Like we mentioned earlier, Cypress will confirm each dialogue by default, meaning if there’s an “Ok” option, it will essentially click that immediately. As Cypress works through these tests, you won’t see that dialogue in the Cypress UI.
Learn more about testing in Cypress and other tools!
To get a step-by-step walkthrough for testing alerts, confirmations, prompts, and a whole lot more in Cypress and even Selenium Java, check out the Automation Cookbook!
The post Testing Browser Alerts, Confirmations, and Prompts with Cypress appeared first on Automated Visual Testing | Applitools.
]]>The post How to Add Visual Testingto Storybook appeared first on Automated Visual Testing | Applitools.
]]>
Creating component and design systems are an important way to make sure you’re delivering consistent experiences to the people visiting your website or application. Storybook makes this easy with its JavaScript API, but how can we make sure we’re tracking the quality of those stories with automated testing?
Just looking for an example? Check out the available Applitools tutorials for React, Vue, and Angular on GitHub.com.
What is Storybook?
Storybook is a JavaScript-powered web tool that gives teams the ability to build component and design systems for their websites and applications.
By adding Storybook to your project, you can develop individual components of your UI in a focused, self-documenting environment, that helps everyone from design to development to product release carefully craft an experience.
It supports a variety of the development community’s most popular frameworks including React, Vue, and Angular. It also has a few documented ways to add testing to those components, but that typically involves meticulously writing tests that could be fragile and not necessarily representing what people are using in their applications.
That’s where Applitools and automated visual testing come in!
How does Storybook work with Applitools?
Pretty much out of the box, Applitools will give you full automated visual testing coverage of all of the components of your Storybook library.
By installing the Applitools Eyes Storybook SDK and running a simple command, Applitools will automatically pull in all of those stories, take a snapshot, and check for regressions in the Applitools cloud.
What are we going to build?
We’re going to start off with an existing React project that already has Storybook installed.
For more information about installing Storybook, visit the Storybook docs at: https://storybook.js.org/docs/react/get-started/install
In this walkthrough, we’ll be using a Storybook React application, but both Storybook and Applitools support more frameworks like Vue and Angular!
But with our project ready to go, we’ll first install the Applitools Eyes Storybook SDK using npm.
Once installed, we’ll show an example of how you can use Applitools Eyes to add Visual Testing coverage to your Storybook.
Step 1: Installing the Applitools Eyes Storybook SDK
To get started with the Applitools Eyes Storybook SDK on a project, first install the SDK from npm:
npm install @applitools/eyes-storybook --save-devThis will pull down the SDK from npm and add it as a package to your project.
Step 2: Running Applitools Eyes for automated Storybook visual testing
Now that our SDK is installed, we actually don’t need to do anything else to get set up, we can dive right into running our tests!
To run our tests, make sure you first head over to the applitools.com dashboard and find your API key, then run the following in your terminal:
APPLITOOLS_API_KEY="[Your Key]" npx eyes-storybook
After that command finishes, you can see that by default, Applitools will go through each story, capture a snapshot, and check for visual differences.
Because this was our first time running our tests, all of our snapshots are new.
If we head over into our Applitools dashboard, we can even see these tests!
We can see that inside Applitools, all of our tests are new and green, which is a good thing. These will serve as the baseline for all of our tests.
Now to see the magic of how Applitools works, we can try breaking one of our components.
Inside Header.js,on line 30, let’s change our user condition to:
{!user ? (We’re essentially switching around the logic, where if this was happening on production, we would want this bug caught!
Let’s run our tests again:
APPLITOOLS_API_KEY="[Your Key]" npx eyes-storybook
We can see this time, we have differences in our tests.
If we go into the Applitools dashboard, not only can we see that we now have Unresolved tests, we can open up one of those tests, and see the difference.
Applitools Eyes was able to see exactly where that issue was and highlight the change, so we can then see that there is a bug in our application!
Inside of the dashboard, we can click the “thumbs down” button, which tells Applitools we reject that change, meaning it failed.
But once we revert our application change and run our tests again, we can see that our tests are now passing again, because they’re working exactly how they should!
What else can you do with Applitools and Storybook?
Ignoring regions
One of the powerful features of Applitools is the ability to ignore specific regions of your visual tests by passing in selectors.
By using the ignoreRegions property inside of your Story’s eyes configuration, you can specify any selector you’d like to ignore in that Story.
Learn more about ignoring regions on npmjs.com
Specifying region types
In addition to simply ignoring regions, you can also set a custom match level for specific regions, such as if you wanted to define a region to include dynamic content, so that Applitools can use AI to understand if the content changed or if the layout changed.
Learn more about configuring regions for content, layout, and more on npmjs.com.
Get Started Today
Want to try out Applitools Eyes with Storybook for yourself? You can get started today at the link below – the account is free forever.
The post How to Add Visual Testingto Storybook appeared first on Automated Visual Testing | Applitools.
]]>The post How to Improve Cypress Test Coverage with Visual Testing appeared first on Automated Visual Testing | Applitools.
]]>
Cypress is a powerful tool for automating tests inside of a browser, but can be limited by only running code-based tests. How can we take advantage of Cypress and add the power of automated Visual Testing with Applitools Eyes?
Just looking for an example? Check out the Applitools Cypress Tutorial on GitHub.com.
What is Cypress?
Cypress is an end-to-end testing framework built on JavaScript that allows QA engineers, developers, or really anyone familiar with working in JavaScript, the ability to write tests that interact directly with browsers.
Tools like Jest might make sense to test the input and output of a function, but you can use Cypress to interact with a button inside of the browser that invokes that function. Taking that a step further, you can automate an entire customer journey using Cypress each step of the way.
How does Cypress work with Visual Testing and Applitools?
Cypress is really good at running tests in the browser, but in order to get effective coverage of that customer journey, you need to write a lot of code with specific selectors that can end up requiring constant maintenance.
That’s where Applitools steps in, adding an additional layer of Visual Testing on top of your tests, combining the interaction automation of Cypress with the AI-powered Applitools Eyes.
When using Applitools Eyes to enable Visual Testing with Cypress, Cypress will still be used to open up the browser, interact with the site, and even visit pages. But, instead of writing lots of assertions for each UI component, Applitools will capture a snapshot of that entire page and upload it to the cloud where it performs its change detection algorithms on your app.
What are we going to build?
We’re going to start off with an existing project that already has Cypress installed.
For more information about installing Cypress, visit the Cypress docs at: https://docs.cypress.io/guides/getting-started/installing-cypress.html#Installing
With our project ready to go, we’ll first install the Applitools Eyes Cypress SDK using npm.
Once installed, we’ll write a new test that covers a basic example of how you can use Applitools Eyes to add Visual Testing coverage to your app.
Step 1: Installing the Applitools Eyes Cypress SDK
To get started with the Applitools Eyes Cypress SDK on a project, first install the SDK from npm:
npm install @applitools/eyes-cypress --save-devThis will pull down the SDK from npm and add it as a package to your project.
Step 2: Setting up the Applitools Eyes Cypress SDK with Cypress
Once installed, the Applitools Eyes Cypress SDK makes configuration easy by making a setup command available.
To configure Applitools Eyes for Cypress, run:
npx eyes-setup
This will do a few things:
- Configure the Applitools Eyes Cypress SDK as a plugin
- Add new Cypress commands to help interact with Applitools Eyes
- Import Typescript definitions to optionally use in the project
But once finished, we’re all set up and ready to write our first Applitools Eyes test! Let’s go ahead and start doing Visual Testing with Cypress and Applitools!
Step 3: Writing a new test with Cypress
To get a good sense of how Applitools works, we’re going to use a demo app with an alternate version that includes a breaking change.
We can create a new Cypress test by adding the following code:
Here’s a breakdown of what’s happening:
- We’re defining a new Cypress test
- Before each test, we’re going to visit our demo app
- In our test, we click the login button to log into the app
- We check that 2 of the on-page headers exist to confirm what page we’re on
And this works great! We’re confident that the process of logging in is working well.
There are two potential problems with this though.
First, we don’t have any existing IDs on the page that allow us to select the specific headers on the page. If that changes the order, it breaks. To alleviate that, we could apply IDs in the code, but even in that case, if someone makes a small change to the text, it breaks.
Additionally, we’re testing specific text nodes on the page that we have access to with code, what happens if an image breaks or if a color changes that makes the app unusable accidentally?
We can test this out by switching the URL we’re visiting in our Cypress test to:
beforeEach(() => cy.visit('https://demo.applitools.com/index_v2.html'));After the test runs, we can see that it was successful, but if we notice on the login page, we have a broken image!
This is where Applitools shines, where we would still be able to detect this issue with Visual Testing.
Step 4: Adding Visual Testing to Cypress with Applitools Eyes
Because we have Applitools Eyes already installed, setting up our test is straightforward.
Let’s update our test to use Applitools Eyes instead of checking specific selectors on the page:
Note: remember to switch back to the original demo for the first run!
In our test, we’re now:
- Opening Applitools Eyes at the start of the test
- Using Check Window to take a snapshot of our Login page
- Logging into our application
- Taking another snapshot with Check Window of our dashboard
- And closing Eyes
Like our first Cypress test, everything looks successful. We were able to log into the app and take a snapshot of both pages.
We can even see these snapshots by logging into our Applitools account.
Now, let’s try changing our demo URL again:
beforeEach(() => cy.visit('https://demo.applitools.com/index_v2.html'));This time, we can see that Cypress actually fails.
Cypress lets us know that it’s because diffs were detected on the Login screen, so let’s take a look in Applitools.
We can see that Applitools found that we have a broken image on our page, so it flagged it as an issue, something our previous Cypress-only test wouldn’t have caught!
While we were able to take advantage of Cypress to run our tests inside of a browser, we were able to effectively use Applitools Eyes to provide automated Visual Testing for our app.
Get Started Today
Want to try out Applitools Eyes for yourself? You can get started today with a free-forever account at the link below.
The post How to Improve Cypress Test Coverage with Visual Testing appeared first on Automated Visual Testing | Applitools.
]]>The post How to Setup GitHub Actions with Cypress & Applitools for a Better Automated Testing Workflow appeared first on Automated Visual Testing | Applitools.
]]>
Applitools provides a number of SDKs that allows you to easily integrate it into your existing workflow. Using tools like Cypress, Espresso, Selenium, Appium, and a wide variety of others, web and native platforms can get automated visual testing coverage with the power of Applitools Eyes.
But what if you’re not looking to integrate it directly to an existing testing workflow because maybe you don’t have one or maybe you don’t have access to it? Or what if you want to provide blanket visual testing coverage on a website without having to maintain which pages get checked?
We’ll walk through how we were able to take advantage of the power of Applitools Eyes and flexibility of GitHub Actions to create a solution that can fit into any GitHub-based workflow.
Note: if you want to skip the “How it Works” and go directly to how to use it, you can check out the Applitools Eyes GitHub Action on github.com. https://github.com/colbyfayock/applitools-eyes-action
What are GitHub Actions?
To start, GitHub Actions are CI/CD-like workflows that you’re able to run right inside of your GitHub repository.
Using a YAML file, we can set up our project to run tests or really any kind of script based on events such as a commit, pull request, or even on a schedule with cron.
name: Tests
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '12'
- run: npm ci
- run: npm testSetting up simpler workflows like our example above, where we’re installing our dependencies and running our tests, is a great example of how we can automate critical code tasks, but GitHub also gives developers a way to package up complex scripts that can reach beyond what a configurable YML file can do.
Using custom GitHub Actions to simplify complex workflows
When creating a custom GitHub Action, we unlock the ability to use scripting tools like shell and node to a greater extent, as well as the ability to stand up entire environments using Docker, which can allow us to really take advantage of our allotted environment just like we could on any other CI/CD platform.
In our case, we want to allow someone to run Applitools Eyes without ever having to think about setting up a test runner.
To achieve this, we can include Cypress (or another test runner) right along with our Action, which would then get installed as a dependency on the workflow environment. This allows us to hook right into the environment to run our tests.
Scaffolding a Cypress environment in a GitHub Action workflow with Docker
Setting up Cypress is typically a somewhat simple process. You can install it locally using npm or yarn where Cypress will manage configuring it for your environment.
This works roughly the same inside of a standard YAML-based GitHub Action workflow. When we use the included environment, GitHub gives us access to a workspace where we can install our packages just like we would locally.
It becomes a bit trickier however when trying to run a custom GitHub Action, where you would want to potentially have access to both the project and the Action’s code to set up the environment and run the tests.
While it might be possible to figure out a solution using only node, Cypress additionally ships a variety of publicly available Docker images which let’s us confidently spin up an environment that Cypress supports. It also gives us a bit more control over how we can configure and run our code inside of that environment in a repeatable way.
Because one of our options for creating a custom Action is Docker, we can easily reference one of the Cypress images right from the start:
FROM cypress/browsers:node12.18.3-chrome87-ff82In this particular instance, we’re spinning up a new Cypress-supported environment with node 12.18.3 installed along with Chrome 87 and Firefox 82.
Installing Cypress and Action dependencies
With our base environment set up, we move to installing dependencies and starting the script. While we’re using a Cypress image that has built-in support, Cypress doesn’t actually come already installed.
When installing Cypress, it uses cache directories to store download binaries of Cypress itself. When using Docker and working with different environments, its important to have predictable locations of where these caches exist, so that we’re able to reference it later.
In our Docker file, we additionally configure that environment with:
ENV NPM_CACHE_FOLDER=/root/.cache/npm
ENV CYPRESS_CACHE_FOLDER=/root/.cache/CypressWhen npm and Cypress goes to install, they’ll use those directories for caching.
We also need to set up an entrypoint which tells Docker what script to run to initiate our Action.
Inside of our Dockerfile, we add:
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]We’ll use a shell script to initiate our initial installation procedure so we can have a little more control over setting things up.
Finally, inside of our referenced shell script, we include:
#!/bin/sh -l
cd $GITHUB_WORKSPACE
git clone https://github.com/colbyfayock/applitools-eyes-action
cd applitools-eyes-action
npm ci
node ./src/action.jsThis will first navigate into our GitHub’s Workspace directory, to make sure our session is in the right location.
We then clone down a copy of our custom Action’s code, which is referencing itself at this point, but it allows us to have a fresh copy in our Workspace directory, giving us access to the script and dependencies we ultimately need.
And with our Action cloned within our working environment, we can now install the dependencies of our action and run the script that will coordinate our tests.
Dynamically creating a sitemap in node
Once our node script is kicked off, the first few steps are to find environment variables and gather Action inputs that will allow our script to be configured by the person using it.
But before we can actually run any tests, we need to know what pages we can run the tests on.
To add some flexibility, we added a few options:
- Base URL: the URL that the tests will run on
- Sitemap URL: this allows someone to pass in an already sitemap URL, rather than trying to dynamically create one
- Max Depth: how deep should we dynamically crawl the site? We’ll touch on this a little more in a bit
With these settings, we can have an idea on how the Action should run.
If no sitemap is provided, we have the ability to create one.
Specifically, we can use a Sitemap Generator package that’s available right on npm that will handle this for us.
const generator = SitemapGenerator(url, {
stripQuerystring: false,
filepath,
maxDepth
});Once we plug in a URL, Sitemap Generator will find all of the links on our page and crawl the site just like Google would with it’s search robots.
We need this crawling to be configurable though, which is where Max Depth comes in. We might not necessarily want our crawler to drill down link after link, which could cause performance issues, but it could also include pages or other websites that we aren’t interested in including in our sitemap.
With Max Depth, we can tell our Sitemap Generator how deep we want it to crawl. A value of 1 would only scrape the top level page, where a value of 2 would follow the links on the first page, and then follow the links on the second page, to find pages to include in our dynamically generated sitemap.
But at this point, whether dynamically generated or provided to us with the Sitemap URL, we should now have a list of pages that we want to run our tests on.
Running Cypress as a script in node
Most of the time when we’re running Cypress, we use the command line or include it as a script inside of our package.json. Because Cypress is available as a node package, we additionally have the ability to run it right inside of a node script just like we would any other function.
Because we already have our environment configured and we’ve determined the settings we want, we can plug these values directly into Cypress:
const results = await cypress.run({
browser: cypressBrowser,
config: {
baseUrl
},
env: {
APPLITOOLS_APP_NAME: appName,
APPLITOOLS_BATCH_NAME: batchName,
APPLITOOLS_CONCURRENCY: concurrency,
APPLITOOLS_SERVER_URL: serverUrl,
PAGES_TO_CHECK: pagesToCheck
},
headless: true,
record: false,
}); If you notice in the script though, we’re setting a few environment variables.
The trick with this is, we can’t directly pass in arguments that we may need inside of Cypress itself, such as settings for Applitools Eyes.
The way we can handle this is by creating Cypress environment variables, which end up roughly working the same as passing arguments into the function, we just need to access it slightly differently.
But beyond some Applitools-specific configurations, the important bits here are that we have a basic headless configuration of Cypress, we turn recording off as ultimately we won’t use that, and we pass in PAGES_TO_CHECK which is an array of pages that we’ll ultimately run through with Cypress and Applitools.
Using Cypress with GitHub Actions to dynamically run visual tests
Now that we’re finally to the point where we’re running Cypress, we can take advantage of the Applitools Eyes SDK for Cypress to easily check all of our pages.
describe('Visual Regression Tests', () => {
const pagesToCheck = Cypress.env('PAGES_TO_CHECK');
pagesToCheck.forEach((route) => {
it(`Visual Diff for ${route}`, () => {
cy.eyesOpen({
appName: Cypress.env('APPLITOOLS_APP_NAME'),
batchName: Cypress.env('APPLITOOLS_BATCH_NAME'),
concurrency: Number(Cypress.env('APPLITOOLS_CONCURRENCY')),
serverUrl: Cypress.env('APPLITOOLS_SERVER_URL'),
});
cy.visit(route);
cy.eyesCheckWindow({
tag: route
});
cy.eyesClose();
});
});
});Back to the critical part of how we ran Cypress, we first grab the pages that we want to check. We can use Cypress.env to grab our PAGES_TO_CHECK variable which is what we’ll use for Eyes coverage.
With that array, we can simply run a forEach loop, where for every page that we have defined, we’ll create a new assertion for that route.
Inside of that assertion, we open up our Eyes, proceed to visit our active page, perform a check to grab a snapshot of that page, and finally close our Eyes.
With that brief snippet of code, we’re uploading a snapshot of each of our pages up to Applitools, where we’ll now be able to test and monitor our web project for issues!
Configuring Applitools Eyes GitHub Action into a workflow
Now for the fun part, we can see how this Action actually works.
To add the Applitools Eyes GitHub Action to a project, inside of an existing workflow, you can add the following as a new step:
steps:
- uses: colbyfayock/applitools-eyes-action@main
with:
APPLITOOLS_API_KEY: ${{secrets.APPLITOOLS_API_KEY}}
appName: Applitools
baseUrl: https://applitools.comWe first specify that we want to use the Action at its current location, then we pass in a few required input options such as an Applitools API Key (which is defined in a Secret), the name of our app, which will be used to label our tests in Applitools, and finally the base URL that we want our tests to run on (or we can optionally pass in a sitemap as noted before).
With just these few lines, any time our steps are triggered by our workflow, our Action will create a new environment where it will run Cypress and use Applitools Eyes to add Visual Testing to the pages on our site!
What’s next for Applitools Eyes GitHub Action?
We have a lot of flexibility with the current iteration of the custom GitHub Action, but it has a few limitations like having an environment already deployed that can be accessed by the script and generally not having some of the advanced Applitools feature customers would expect.
Because we’re using node inside of our own custom environment, we have the ability to provide advanced solutions for the project we want to run the tests on, such as providing an additional input for a static directory of files, which would allow our Action to spin up a local server and perform the tests on.
As far as adding additional Applitools features, we’re only limited to what the SDK allows, as we can scale our input options and configuration to allow customers to use whatever features they’d like.
This Action is still in an experimental stage as we try to figure out what direction we ultimately want to take it and what features could prove most useful for getting people up and running with Visual Testing, but even today, this Action can help immediately add broad coverage to a web project with a simple line in a GitHub Action workflow file.
To follow along with feature development, to report issues, or to help contribute, you can check out the Action on GitHub at https://github.com/colbyfayock/applitools-eyes-action.
The post How to Setup GitHub Actions with Cypress & Applitools for a Better Automated Testing Workflow appeared first on Automated Visual Testing | Applitools.
]]>The post Next.js Visual Regression Testing with Applitools and Cypress appeared first on Automated Visual Testing | Applitools.
]]>
Visual regression testing is an effective way to make sure your app is behaving the way you want it no matter the technology stack. As someone who comes from the JavaScript and Jamstack world, I want to be able to use modern tooling to make sure my dynamic web apps are reliable and maintaining a high level of quality.
I recently joined Applitools as a Developer Advocate. With my role, it will be crucial for me to have a good understanding of not just how to install Applitools and the suite of features we provide, but I’ll need to be able to thoroughly understand how it fits into a developer’s workflow from installation to implementation.
In this stream, I take my first dive into building a new React app from the ground up with Next.js. I then walk through installing JavasScript testing framework Cypress, giving me the ability to run end-to-end testing for my new app. Finally, I install and set up Applitools Eyes which provides my app visual regression testing capabilities baked right into my testing suite.
To learn more about how visual regression testing with Applitools Eyes works, check out our Automated Regression Testing with Visual AI page.
The post Next.js Visual Regression Testing with Applitools and Cypress appeared first on Automated Visual Testing | Applitools.
]]>