80+ questions and answers about one of the
hottest new testing frameworks around, Playwright!

Playwright is a relatively new open source cross-browser automation framework for end-to-end testing, developed and maintained by Microsoft. It tests across all modern browsers and is designed to be a framework that solves the needs of testing for today’s web apps.

Andrey Lushnikov, Principal Engineer at Microsoft, recently spoke with us in a webinar covering the inspiration for the development of Playwright, numerous features and efficiency tricks, as well as its cross-browser availability and support for multiple languages.

You can check out the full webinar, Playwright: A New Test Automation Framework for the Modern Web, at the link below.

There was a lively Q&A component to the webinar, and there wasn’t time to answer all the questions live. So the team at Microsoft looked at the remaining questions (all 80 of them!) and answered them. We’ve collected them all here for you so that, between the webinar and this extended Q&A, you can get all your Playwright questions answered in one place.

If you have a question that’s not answered here, you can also feel free to drop it on the Playwright Slack. Let us know about it @Applitools and we’ll be sure to add it to this post.

Extended Q&A: Your 80 Questions about Playwright

Click to skip to a section:

• Features Questions
• Integration Questions
• Migration Questions
• Platform Questions
• Availability Questions
• Comparison Questions
• Design Questions
• Support Questions

Features Questions

Q: Is there any built in reporting in Playwright?

A: No, reporting is typically used on the test runner level. You can use existing reporting tools along with the existing test runners with Playwright. Having said that, we are working on our own test runner for Node to address some of the existing e2e testing pain points.

Q: Can Playwright discover UI performance issues, for example, if a drop down list takes some time to load the mouse movement is not being followed on the items in the list, is that something that Playwright can help with discovering?

A: Here’s a quick overview of our performance story:

• On the high level, you can measure the time your calls took. We are working on a playwright trace that would have this information in a serialized and machine-processable form. That’ll happen ~v1.12, in a month.
• For Chromium, you can use tracing to drop this file into devtools and get in-depth performance story: https://playwright.dev/docs/next/api/class-browser#browserstarttracingpage-options this link is to the new doc, but the method has been there since forever.
• We expose resource timing https://playwright.dev/docs/api/class-request#requesttiming

I think Java will only catch up with chrome tracing API starting with v1.11. JavaScript and Python already have it.

Q: Does it support mocking? Like cy.intercpet? Can we get a coverage report from playwright? And can we wait like cy.wait()? And can we run a webkit test in container as part of pipeline?

A: The answer is yes – you can see more here: https://playwright.dev/docs/network#handle-requests

Q: Does Playwright provide access to windows displayed by the browser itself, not by web pages? (E.g. File Download/Upload windows)

A: Yes! Here’s more about Downloads / Uploads. Playwright exposes APIs to accept / decline / process downloads as well as for uploads. You can handle file upload dialog as well as set files into the input element.

Q: When a page is opened using browser context, is the page rendered in GUI or opened headlessly? In other words – what is the chance that rendered page will differ for the human when compared to test code?

A: There is no difference between headless and headful in terms of rendering.

Q: How does Playwright handle Salesforce iFrames?

A: If this is about new out-of-process iFrames in Chromium-based browsers, yes, we support them!

Q: How long does Playwright wait before giving up on waiting for an element?

A: It waits for a given timeout (can be local or global, 30 sec default). Check out this link for more: https://playwright.dev/docs/actionability

Q: Can Playwright identify elements by ID?

A: Yes, and many more – see more about our selectors here: https://playwright.dev/docs/selectors

Q: Can the tests run in parallel?

A: Yes, test runners like pytest and Jest will parallelize Playwright-based tests.

Q: Is it possible to show the cursor on a video recording?

A: Not at this moment, but please request it via GitHub.

Q: Are these assertions [demonstrated in the webinar] native to Playwright? or are they from Jest library?

A: Examples are shown using expect library (Jest). But that’s really up to you.

Q: How do you implement Data Driven methods? How do you get external data into script and drive multiple iterations with different data sets?

A: That’s more of a test runner question. Different test runners offer different APIs for that.

Q: Does Playwright offer a special functionality to handle canvas?

A: Nothing special for canvas. You can click with position to do free hand input there, but that’s not specific to canvas.

Q: What if I want to click a button that is disabled? What is Playwright’s wait strategy – does the test run into a timeout?

A: It’ll wait until the timeout: https://playwright.dev/docs/actionability.

Q: How do you run a set of test cases on different files?

A: That would be defined by the test runner you use: jest-playwright, cucumber, etc. We are working on a test runner for Playwright ourselves, which we will talk about a little later.

Q: Does an element need to be in the viewport to be active?

A: We will scroll to the element. And make sure it is not occluded by the sticky header.

Q: Do we get verbose logs saying what Playwright was trying to do before any test failure?

A: Yes you do – you can read more about verbose logs in Playwright here.

Q: Can we see what errors look like when a test fails?

A: They will be in the form of exceptions, but those exceptions contain verbose logs of activities.

Q: Does playwright provide some support to easily achieve multithreading?

A: Nothing special for multithreading. The test runner will shard testing processes to achieve concurrency.

Q: On Auto-waiting – how does this work? If, for example, a button gets enabled after a request comes in it needs to check this continuously? So you would need some kind of probing mechanisms that check every x seconds and timing out after y seconds?

A: Something like this. It will be rAF on log time scale probes, but happening on the browser side for perf reasons.

Q: In the demo [shown in the webinar], there is a waitForNavigate method. What exactly is implemented in waitForNavigate?

A: Put simply, waitForNavigation will wait for any navigation to happen.

Q: Why is waitForNavigation needed if Playwright has an auto-wait mechanism?

A: It is there for when you might need it, for example to make it absolutely clear that you are expecting a navigation. Say for example you’re filling a multi-page form and each page has a “next” button that brings you to the next page. If you want to click “next” on the first two pages, you might use waitForNavigation after the first click and before the second one to make sure you don’t click the same first button twice.

Q: What about automation animated elements, for instance pages with WebGL. Does Playwright support that kind of interaction?

A: As far as Playwright is concerned, WebGL is just a canvas element, you can interact with it using the mouse and script it using JavaScript. There is no specific support for WebGL vs regular canvas.

Q: Do you plan to include something for component testing at some point?

A: That’s a very interesting and hot topic. For those interested in this, please file an issue here for us to learn more about your use case.

Q: How easy is it to run parallel test against other tools like Cypress?

A: Test runners will do that seamlessly, you don’t need to do anything.

Q: Do you recommend Playwright for API testing?

A: That depends. We are seeing customers using Playwright for API and load testing when they need real user patterns, but in most of the cases you are better of with the dedicated API testing tools.

Q: Is it mandatory to use geolocation? If so, how do we get a position?

A: No, you can set it to test geo behavior, but you don’t need to.

Q: How does Playwright handle (or bypass) any SSL cert/protocol error?

A: We hook into the browsers and tell them to trust any TLS/certificates you wish. Check out the docs: https://playwright.dev/docs/api/class-browser?_highlight=ignorehttps#browsernewcontextoptions

Q: Does Playwright auto detect page ready?

A: There is no such thing as a page ready in general terms. We encourage you to keep things visual, you say page.click('text=Submit') and you don’t really need to wait for page to be ready, all you need to wait for is an interactable ‘Submit’ button. That makes the life of test writing so much easier.

Q: Can I wait for, intercept and parse HTTP requests (e.g. API calls) happening in the background, when page is loaded or is changed?

A: Absolutely: https://playwright.dev/docs/network#handle-requests

Q: Is multi-user testing (e.g. shared Excel access at the same time) with updating/data integration between two users possible?

A: I think I misunderstood this question during the live session. If two different users that have the same web page open, then absolutely, you can modify page on behalf of one user and see results in the other.

Q: Does Playwright by default work asynchronously? That is, do we not have to use async/await explicitly?

A: That depends on the language. Our Java is sync API, our Python can be both sync and async, our JavaScript is async. We try to make it fit the ecosystems and be idiomatic.

Q: How do you use Playwright Inspector when working in Visual Studio Code?

A: We would expect that you would typically use one or the other, depending on the use case.

Q: What’s an alternative of cy.click({force:true}) in Playwright ? Is it integrated in the Auto Wait feature?

A: It’s page.click({ force: true }). For more, see: https://playwright.dev/docs/api/class-page#pageclickselector-options

Q: Sometimes you need to wait for page to be fully loaded before taking a screenshot. For SPA, waitForNavigation and waitForLoadState do not work well. Is waitForSelector the best choice for this situation?

A: Yes, you nailed it, there is nothing better than a manual page ready criteria based on something visual.

Integration Questions

Q: What about reporting in Playwright? Does it have a CLI to aggregate reports from multiple runs? A reporting dashboard?

A: We don’t (yet!) ship a test runner with Playwright. We assume you will use your favorite runner and utilize its reporter.

Q: How does Playwright work with Continuous Integration?

A: You can run Playwright with your favorite test runner on your favorite CI: https://playwright.dev/docs/ci

Q: Can we integrate unit test libraries such as JUnit and TestNg with Playwright?

A: You can use Playwright with any of these unit testing frameworks! That’s in fact what most of our customers do.

Q: How easy it is to integrate parallel testing solutions like zalenium?

A: I am not familiar with zalenium, but parallel testing is a priority for us. We make sure all test runners can run Playwright in parallel.

Q: Can we integrate Playwright with visual tools?

A: Applitools Visual testing has a SDK for Playwright here. [Editor’s Note: We’ve also got a blog post with a great example of using Playwright with visual testing here]

Q: Will there be an integration with Cucumber?

A: There already is one – you can read more about it here: https://tally-b.medium.com/e2e-testing-with-cucumber-and-playwright-9584d3ef3360

Q: Will there be integration with a test management tool, such as TestRail?

A: That would need to happen on the higher level (test runner level).

Q: Does playwright come with a test explorer for Visual Studio Code?

A: That would need to happen on the test runner level too.

Q: Is there a GitHub Action for Playwright cloud execution?

A: Yes! You can read more about Playwright and GitHub Actions here: https://playwright.dev/docs/ci#github-actions

Q: Are Playwright assertions unique? Does it depend on a separate assertion library like Chai?

A: Playwright does not assert, you use it with your preferred library. You can use Expect or Chai, or anything else.

Migration Questions

Q: In our organization we use Jest-Puppeteer-Cucumber (TypeScript). Is there anything in Playwright that could help ease a possible migration from that tech stack?

A: There is no migration tool / guide, but the APIs for basic operations are very similar. You’ll end up deleting a bunch of waitForSelectors and then it’ll work as is.

Q: In terms of migration, would we need to rework our current Selenium WebDriver-based framework and redo a new one based on Playwright?

A: If you are using modern CI/CD, Playwright will work out of the box, for example using GitHub Actions. But if you have on-premise Selenium grid, you would need your current framework to be extended to allow pointing Playwright clients to your browser farm. Then you can gradually migrate.

Platform Questions

Q: Does Playwright have any plans for mobile apps? Native and Hybrid?

A: Experimental support for Android is available: https://playwright.dev/docs/api/class-android, both browser and WebViews. Nothing for iOS so far. Please star the issue on GitHub and share your requirements there.

Q: Is there an open source scale-out tool like Selenium Grid that will work with Playwright? If not, are there plans to build one?

A: Our APIs allow third parties to run services providing Selenium Grid-like features with Playwright. We are not commenting on whether we are building one.

Q: How does Playwright talk to the browser? With an HTTP request?

A: Playwright uses browser remote debugging protocols (e.g. CDP for Chromium) over the process pipe.

Q: Does Playwright support the Safari browser?

A: Yes, Safari is based on WebKit, one of the three rendering engines we support. Learn more at https://playwright.dev/docs/browsers.

Q: Are there any browser version restrictions for Playwright?

A: Playwright supports the latest versions of the browsers: their stable, beta channels for Chromium and Firefox-based browsers. Our WebKit corresponds to the Safari Technology Preview version. Learn more at https://playwright.dev/docs/browsers.

Q: How does Playwright manage multiple testing over multiple browser versions ? Ie, chrome 88, 87, etc.

A: Google Chrome and Microsoft Edge are trying hard to not allow downloads of the stale browsers and force-update their users. We keep Playwright in sync with the latest, but you can always fetch an older version of Playwright that works with say 86.

Q: Why is the Chromium browser launched in incognito mode?

A: For browser context isolation, we are using an internal concept called off-the-record-profile. It is rendered as incognito, because it is, well, off the record

Q: Is there any target to have a complete framework that covers web, mobile and desktop testing? And what about cloud testing?

A: We are only focusing on web everywhere at this point. Playwright does also have experimental Android and experimental Electron.

Availability Questions

Q: Is Playwright an open source or license-based library?

A: It’s open source.

Q: Does Playwright support Node.js and JavaScript right now?

A: Playwright supports JavaScript, TypeScript, Java, Python, Python asyncio, and C# in alpha. Learn more at https://playwright.dev/docs/languages

Q: Can you tell me more about the Playwright C# implementation?

A: You can find more here: https://github.com/microsoft/playwright-sharp. Again, it’s in alpha state as of now.

Q: Is there anyway to get the codegen to write ES6 JavaScript?

A: We only generate ES5 style code now, but it can be added upon request.

Q: Will codegen support TypeScript?

A: The ES5 code that we produce is valid TypeScript. We don’t have methods in the generated snippets, so there isn’t much room for types anyways.

Q: Does Playwright support JavaScript?

A: Yes, the very first version of Playwright was for JavaScript and TypeScript. Learn more at https://playwright.dev/docs/intro.

Q: Though it supports multiple languages, which is the recommended language for Playwright?

A: The one you are comfortable with. Our Node.js version was the first one, so if you don’t care – pick the JavaScript one.

Q: We have various frameworks like Cypress, TestCafe, Jest/Puppeteer, Playwright and many more. How should we decide which framework to pick because at the end of the day all are trying to achieve speed with respect to selenium?

A: I’d take a look at Playwright’s API (picking just one class from this API: https://playwright.dev/docs/api/class-page), the set of browsers we support and the test coverage we have for it. [Editor’s Note: You can also check out a great comparison of these tools here.]

Q: Any support for Dart in the future?

A: Please file a request on GitHub. If it gets enough stars, we will add it.

Q: Can you convert code snippets? For example JavaScript to Python and vice versa?

A: No, we can generate code in all languages, but not convert between languages.

Q: My question is related to building an enterprise framework for test automation using Java. How will Java Playwright be developed in future? Will it solve the limitations of Selenium for a JavaScript based application?

A: Playwright is already many times more powerful than WebDriver-based solutions, and Java version is on par with the rest of the Playwright family. Our primary goal is to keep resolving those limitations.

Q: If APIs are implemented in C#/Java would it be ok to use JavaScript for the test side for Playweight? Or would it make sense to run under same solution and implement in C#/Java?

A: That’s completely your call. Many people use JavaScript and Python for programs written in Java/C#. Language is a tool in your hands, you pick the one you feel more comfortable.

Q: What is better with Playwright? Jest or Folio?

A: Jest-playwright is great, but Folio will be absolutely awesome. We are very close to releasing it to public, please stay tuned. Don’t use Folio just yet, but it is coming really soon.

Q: Does Playwright release on a schedule that coincides with third party dependencies such as WebKit, Chrome upgrades, WebDrivers etc?

A: We release monthly.

Comparison Questions

Q: How does this differ from TestCafe?

A: The feature set is vastly different. For example, our page can do the following things: https://playwright.dev/docs/api/class-page

Q: There seems to be a movement to no-code automation. How does playwright stack up against those vendor products?

A: Playwright is a foundation. Its APIs bet on the script-based testing, but in reality it just makes so much possible when it comes to cross-browser testing. I can imagine a lot of third party no-code solutions emerging that would utilize Playwright as a foundation.

Q: How is Playwright different from other tools like Cypress?

A: Playwright supports all rendering engines, it has more capabilities (network, proxies, mobile, emulation, many more) because we hook into the browsers directly. We also have a traditional programming model (instead of chained API).

Design Questions

Q: Do you use an equivalent to Page Object Model? I need to try to avoid hard coded values for objects and other items.

A: Playwright script is a program, so you can treat it as such. As in any program, you don’t hard-code, but use variables, parameters, etc. You can do the same with Playwright.

Q: Is there any specific test automation design pattern you suggest to use with Playwright? Like Page Object Model was most popular with Selenium.

A: Yes the Page Object Model design pattern works well with Playwright, as do alternative design patterns. It’s really up to you. [Editor’s Note: We’ve got a great blog post demonstrating Page Object Model with Playwright]

Q: From an architecture standpoint, can we apply the Page Object Model, or Screenplay?

A: Playwright is just a library, you can build any programs on top of it using any design patterns.

Q: Can you please help me understand how to implement the Page Object Model using JavaScript, and how to generate an executable?

A: For Node.js, you don’t generate an executable, you typically write a program and then run it with ‘node script.js’. But you don’t need to use Node.js if that is not your preferred language, Playwright for Java / Python is just as great.

Support Questions

Q: Where is the best community support source for Playwright?

A: The best place to go is our Slack channel, which you can find here: https://aka.ms/playwright-slack

Q: Is there any forum for Playwright Java which can be referred to raise issues or queries?

A: Head over to the Playwright Slack and pick the -java channel. Or just file an issue to Playwright Java on GitHub.

Q: Who “owns” Playwright? Microsoft? Will it be supported for a long time?

A: Playwright is developed in the Developer Division at Microsoft. That’s the same people that brought Visual Studio, Visual Studio Code, .NET and Azure tools to you. Developers are our mission, our core business. It is not a side library that we did for fun, we are here to stay.

Q: How can we be confident this framework will last? Do you have a long-term support plan?

A: This is a core business for our division, we’d like to see more testing that can run in the cloud. Everything that runs in the cloud is great for Microsoft as well. You’ll actually see more Playwright-related announcements and deliverables coming from us.

Thank You

We wanted to take the chance again to say thank you to Andrew Lushnikov and Pavel Feldman of Microsoft for leading the Playwright webinar that inspired all these questions and providing all these answers. And of course, thanks to everyone who participated in the webinar and asked these thoughtful questions.

If you’re looking to do visual testing alongside Playwright, be sure to check out the Applitools SDK for Playwright here. Happy testing!

The post Your Favorite Questions about Playwright, Answered From The Applitools Team appeared first on Automated Visual Testing | Applitools.

For anyone trying to automate repetitive and time-consuming tasks, Microsoft has just quietly released a free tool that will make your life easier. Power Automate Desktop is a codeless tool that arose out of their Softomotive acquisition a year ago. Now fully integrated into Windows, it is available for free to all Windows 10 users.

In this step-by-step tutorial, we’ll learn how to use Microsoft Power Automate Desktop to automate an application without code. We’ll even learn how to codelessly test it visually too.

Automating with Power Automate Desktop

So what can be done with Power Automate Desktop? A better question might be, what can’t you do with it? Power Automate Desktop provides many already pre-built “Actions” to automate many different processes or applications out of  the box. For example, you may want to automate sending an email triggered by a particular event. Or you may want to extract values from an Excel spreadsheet and store them somewhere else like a database on Azure. If so, Power Automate Desktop has you covered along with hundreds of other similar actions. If an action you need doesn’t exist, no problem either. It’s easy to create one – so let’s go ahead and do so! 

A Codeless Example of Power Automate Desktop

For the context of this example. Let’s say you’re a tester and you’ve been tasked to automate your Windows desktop application. Where do you start!? You could go down the path of using a framework such as WinAppDriver, which is an Appium driver built and maintained by Microsoft to automate desktop Windows applications. However, with this approach you will need some coding experience and some knowledge of the framework to really provide a viable solution. There are other enterprise options such as Micro Focus’s UFT One, formally named QTP/UFT. But this too requires some coding knowledge to be successful, along with a hefty paid subscription. 

The beauty of Power Automate Desktop is that, not only is it completely free, but it can be 100% codeless and no coding knowledge or background is required. A Web and Desktop recorder is built in to record any UI elements or views you need to interact with for your automation, which can easily be plugged into your “Flow.” A Flow in Power Automate Desktop is essentially the same thing as “automation script” in the coding world. 

For this example we are going to use a very simple application: the Calculator app. 

Creating Your First Flow

1. If you haven’t done so already, download and install Power Automate Desktop.
2. Open Power Automate Desktop and create a New Flow.

   

3. This will open up the Action flow builder screen.
4. Let’s choose for our first action to “Run application” and set it to open the Calculator app.

   

5. We want to give it some time to wait for the Calculator application to start. There are many different wait actions we can use. I unfortunately didn’t have a lot of luck using some explicit waits so I used an implicit 5 second wait instead that gave me the desired result.

   

6. Next, let’s take a screenshot of the calculator application and store it in a folder in PNG format. We will use the “Take screenshot” action.

   Tip! Whenever you screenshot your application, make sure the application window is isolated (preferably with a dark background) and not overlapping any desktop icons or other windows. This will provide a much better image result.

   

7. We’ll now click on an element on the Calculator application. To do this we can use the action “Click UI element in window” and the built-in recorder by clicking the “Add a new UI element” button to detect an element, in this case the 8 button, to click. When hovering over the 8 button, you can use Ctrl+left click to store the element in your list. Set the Click type to Left click.

   

   

8. Go ahead and repeat the step above by adding 3 more “Click UI element in window” actions to multiply 8×8=64.

   

9. Add one more screenshot action (like in step 6 above). Give the screenshot filename something different than you did in step 6 and store it in the same folder. 
10. Finally, use the action “Close window” to close the Calculator application.

    

11. Now go ahead and execute your Flow by clicking the Run button! If all goes to plan, you should see the Calculator application launch, take a screenshot, multiply 8×8, take another screenshot and finally close the Calculator application.
12. Your two screenshot images should look similar to this:

    

Visually Validating Your Desktop App

So now let’s say you want to visually validate your desktop application with Applitools using our AI image comparison algorithms. But why would you want to visually validate your application? 

Why Visually Validate Your Application? 

Well the old saying goes “A picture is worth a thousand words.” That cannot be any truer than when it comes to testing. Factor in that in today’s software development world, the pace of releasing software updates is drastically increasing. A visual snapshot of your application can uncover bugs, visual discrepancies or layout issues you never knew existed and also ensure your Application’s UI still works and is functional. 

Like in this example, if 8×8 didn’t equal 64 we’d know right away something was wrong because the screenshot would look different. With an image, every visual attribute (text, color, location, etc..) of every element is automatically captured. 

Visual Testing with Power Automate Desktop

So lets add an action to upload our screenshots we took above to Applitools!

1. For this next step we have a few different options. Applitools provides over 50+ SDKs to do visual testing. You can integrate these SDKs directly into your automation framework or use one of our standalone codeless tools. Power Automate Desktop does provide actions to execute programmable scripts such as Python. However, for this example we are going to use our ImageTester CLI codeless tool (also known as the Screenshots CLI) to upload these screenshots from the command line. 
2. If you haven’t done so already, download the ImageTester and place it in a folder of your choosing on your PC. 
3. Next add your Applitools API Key to your System Environment Variables as APPLITOOLS_API_KEY.
4. Now we want to add another action called “Get environment variable” and place it after the “Close window” action.

   

5. Finally, let’s add one more action, “Run DOS command,” to execute the ImageTester CLI we downloaded above. This is where we’re going to use the API Key we are storing in the EnvironmentVariableValue above.

   

6. Your final Flow should look similar to this.

   

7. Now let’s run our Flow again! After this execution we should see our images in Applitools as new baselines.

   

Testing that is Automated and Visually Perfect

And that’s it! You now have stored baseline images of your application. So whenever your development team makes changes to your application in the future you can run this Power Automate Desktop Flow again to ensure your application still functionally works the same but is also visually perfect!

You can see the full Flow in action in the video below:

Applitools comes with many amazing features such as CI/CD integrations, bug tracking and accessibility testing, just to name a few. Visual testing with Applitools will help you improve your overall software quality, test smarter and more efficiently, and release application updates faster without sacrificing quality. Applitools is free so join Applitools today and see for yourself. 

And don’t forget to download Power Automate Desktop and play around with it now that it’s available. Happy testing!

The post Tutorial: How to Automate with Power Automate Desktop appeared first on Automated Visual Testing | Applitools.

Coding recipes using the newest automation testing tool

There’s a new kid on the block in the world of automation testing tools….Playwright!

Created by Microsoft, Playwright is an open source browser automation framework that allows JavaScript engineers to test their web applications on Chromium, Webkit, and Firefox browsers.

While Playwright is still very new, there’s quite a buzz around the tool. So, I decided to try it out for myself.

The Playwright documentation contains a ton of information about how to get started as well as details on their APIs. So, I won’t rehash all of that here. Instead, I will automate real scenarios from the Automation Bookstore application, which will demonstrate examples of using the Playwright APIs together.

Launching an Application

First things first…launching the browser and navigating to the application’s url.

The first step to do this is to declare which browser engines you’d like to run against. Again, Playwright allows you to run your tests against Chromium, Firefox, and Webkit.

I’ll only run against Chromium for now.

Sidenote: By default, Playwright runs in headless mode. If you’d like to actually see the browser as it’s executing the test, set headless to false in the launch call:

Next, I declare variables for the browser and the page objects.

An important thing to note about Playwright, is that all of their APIs are asynchronous, so async/await is needed to call them. So, I utilize this pattern to launch the browser and initial the page object.

Now that I have the page object, I can go to my application using page.goto()

Adding an Assertion Library

A key thing to note about Playwright, is like many other automated testing tools, it is designed to automate browser interaction, but you must use an assertion tool for your verifications. I’ll use Mocha for this example.

I’ll also add a suite (using ‘describe’) and move the Playwright calls inside of there into before/after functions.

Accessing Elements

When trying a new tool, I like to start with a very simple example. Basically, the “Hello World” of web test automation is verifying the title on the page. So, let’s start there and make the first test!

Playwright offers many ways to access elements including the typical ones of CSS and Xpath selectors.

When inspecting the DOM of this application, the element that displays the title has an id of ‘page-title’, and the text I want to verify is the inner text of this element.

So, I’ll use page.innerText and pass in the CSS selector for this id.

And voila, just like that, I have my very first test!

To run a Playwright test, type npm test from your terminal.

Entering Text

Now that I’m up and running, I’d like to do something a little more interesting, such as search for a book.

To do so, I need to enter text into the Filter Books field (which has an id of ‘searchBar’).

To enter text, Playwright provides the fill() API, which accepts a selector as well as the text to enter.

Waiting for Elements

Playwright does a great job of waiting for the elements to be ready before attempting to interact with them, so I don’t have to worry about that. However, sometimes explicitly waiting is still needed.

 For example, when I enter text into the field, Playwright will wait for the field since that’s the element we’re interacting with, but the verification needs to occur on the search results.

There is a slight delay between the time that the text is entered and when the results are shown, so my script must account for that.

Fortunately, in the latest release of Playwright (1.4), the waitForSelector API was introduced. I can use this to wait until hidden books become present in the DOM.

Getting List of Elements

Now that I have the search results, I want to verify that there is only one book returned. To get a list of the elements, I use the $$ API on the page object to query for all elements matching a given selector.

Now, that I have this list of elements, I can verify that it only contains one book.

Accessing Children Elements

In addition to verifying the quantity, I also need to verify that the title is as expected. Since I have a list of all of the visible books, I don’t know exactly what the ID will be for the book at runtime, so I’d prefer to just get the child element from the visible book that holds the title. This is an h2 element, as seen on line 4.

To access descendant elements in Playwright, you can use Clauses. Clauses are selectors that are separated by >>, where each clause is a selector that is relative to the one before it.

So, in my case, where I’d like to get h2 that is a child of this particular li, I can do so with ‘li:not(.ui-screen-hidden) >> h2’

And now the second test is done! On line 5 is where the parent selector is defined, and line 8 is where I expand to use a clause to target a child node.

This particular scenario used a list that only contains one element, so I added one more scenario that demonstrates how to work with a list of multiple elements.

Visual Testing

That last scenario got a bit long, and honestly, I’m only verifying a fraction of what I should be verifying. For more thorough tests that not only verify the count and title, but also verify everything else on the page in addition to making sure it’s displayed correctly, I’ll use visual testing.

Note that Playwright offers image and video capture, but it’s more meant as a visual logging mechanism, and can’t be used in the assertions themselves. So, I’m going to use a proper visual testing assertion library, Applitools.

After installing Applitools, I specify the Applitools classes required on line 3.

I declare and initialize the ‘eyes’ object (on line 3), which is the API that does the visual testing.

Then redo the last test using visual testing.

Cross Platform Tests

While Playwright provides support for Chromium, Webkit, and Firefox, there is no built-in support for IE.

When integrating Playwright with Applitools’ Ultrafast Grid, I can now get further coverage for my tests!

As opposed to using the Applitools Classic Runner as I have above, I’m going to modify it to use the Visual Grid Runner. On line 5, notice I use the number 10. This indicates the number of tests I’d like to run in parallel…making my run even faster!

Now for the good part! I can specify all of the browsers, devices, and viewports I’d like to run against – giving me the ultimate cross-platform coverage for my tests. Not only verifying them functionally, but also visually as well.

No changes are needed to the tests themselves, they will automatically run across all of the specified configurations.

Play with Playwright Yourself

Reading blog posts helped me understand what Playwright is but when I played around with it to explore realistic scenarios is when I truly got a feel for the tool! I recommend you do the same.

You can use the code examples from this blog post as well as Playwright’s documentation to get started. Good luck!

The post Playing with Playwright appeared first on Automated Visual Testing | Applitools.

We created Test Automation University (TAU) to help you learn best practices for using test automation in your app development process.  As we have added classes to TAU, it became apparent that we needed to make our site more productive for users with different interests. So we included learning paths.

Learning paths give you a way to find the courses right for you. Are you writing test automation for APIs in Java? We can send you down the right path. How about testing mobile devices with Swift? Check. What about web tests with Ruby? Yep.

No matter what kind of tests you are trying to automate, you can find a learning path to help you.  For each track, you start with the basics. Once you finish a path, you have learned core concepts and practiced some of the critical skills needed to automate your testing.

With the new learning paths, TAU makes it easy to develop the skill set you seek.

Choosing a learning path is pretty straightforward. On the main page showing all the Test Automation University courses, select the Learning Paths tab:

Unfiltered, this page shows all paths for all languages. The selector lets you filter the learning paths by path type or by language.

As one of the early users of Test Automation University, I appreciate the new learning paths. They have helped me understand the skills I need to develop in software testing – and I love the hands-on nature of the courses.

Let’s go through the paths to give you a sense of what each provides you.

Learning Path Overview

Learning paths through TAU are broken into general functional test areas:

• Web UI
• API Testing
• Mobile
• Codeless

Codeless is, as you would expect, a path that needs no direct coding skill.  Otherwise, the paths let you select one of six languages on which to focus your testing skills:

• Java
• JavaScript
• Python
• C#
• Ruby
• Swift

We continue to develop these paths, so expect changes to both this blog post and to TestAutomationU going forward.

How Do I Do Web UI Testing With Java?

To answer this question, let’s select the Web UI Path, and then select Java.

Let’s take a look at the courses today in the Web UI Java Path.

• Setting a Foundation for Successful Test Automation
• Java Programming
• Web Element Locator Strategies
• Introduction to TestNG
• Automated Visual Testing: A Fast Path To Test Automation Success
• The Whole Team Approach to Continuous Testing
• Scaling Tests with Docker
• Test Automation in DevOps
• Testing From The Inside: Unit Testing Edition
• Tools and Techniques for Performance and Load Testing

These ten classes involve a little over 17 hours of instruction. Each course dives into different elements.

Two of these courses apply to the Java Language specifically: programming in Java and the Java automation engine. Two more are common among the web UI test path: finding web elements on a page, and how to test visually. The others are part of the common core courses shared among all the testing paths.

Common Core

Six courses are generally common across almost all the learning paths. These courses answer key questions for understanding and deploying automated tests. These courses and their associated questions are:

• How do I think about test automation? 
  Course: Setting a Foundation for Successful Test Automation
• How do we achieve continuous testing?
  Course: The Whole Team Approach to Continuous Testing
• How do we scale our testing efforts to ensure fast feedback?
  Course: Scaling Tests with Docker
• How do we automate our tests in the culture of DevOps?
  Course: Test Automation in DevOps 
• How do we test as close to the production code as possible?
  Course: Testing From The Inside: Unit Testing Edition
• How do I automate load testing and performance testing effectively?
  Course: Tools and Techniques for Performance and Load Testing

The questions these courses answer are easily transferred to any combination of languages and test cases.

How Do I Do API Testing With Java?

Let’s take a look at the Java API course.

The Java API Testing path looks like this:

• Setting a Foundation for Successful Test Automation
• Exploring Service APIs through Test Automation
• Java Programming
• Automating your API tests with REST Assured
• The Whole Team Approach to Continuous Testing
• Scaling Tests with Docker
• Test Automation in DevOps
• Testing From The Inside: Unit Testing Edition

This path delivers two API-specific test courses.  We include Amber Race’s course on Exploring Service APIs through Test Automation to introduce API test automation concepts. We share Bas Dijkstra’s course on using REST Assured for REST API test automation to provide alternatives for API testing. The Java course teaches Java. All the other courses are part of the core.

Once you know the core, learning a new language and technology is simply a matter of taking the relevant language and technology courses.

Future Courses

As we continue to add learning paths to TAU, our core goal is to answer these questions:

• How Do I Do UI Testing?
• How Do I Do API Testing?
• How Do I Do Mobile Testing?
• How Do I Do Codeless Testing?

What is on the horizon for Test Automation University?

I am writing this post early in September 2019, and I expect we will update this post regularly, as we continue to add courses to Test Automation University.

In the immediate term , we just launched the course Introduction to Cypress, which shows how developers and testers can use Cypress to test their web applications. We have added this course to the Web UI JavaScript path as an alternative to using webdriver.io.

Keep checking out Test Automation University for the latest offerings.

Find Out More about Applitools

As the sponsors of Test Automation University, we’re excited to help you get even more efficient in your UI testing.  It’s all up to you to find out more.

You can request a demo of Applitools.

You can sign up for a free Applitools account.

You can check out our tutorials.

The post New Learning Paths through Test Automation University appeared first on Automated Visual Testing | Applitools.

We are looking forward to participating in Microsoft Western Europe Digital Success Academy 2019 (DSA WE 2019) at the Rome Convention Center next month.

What is Microsoft Digital Success Academy (DSA)? Microsoft’s DSA is a readiness event for Western Europe Microsoft Sellers; focusing on strategy and aligning with partner offerings.

This year’s DSA will bring together over 2,000 Microsoft Western Europe executives and sellers in one location to learn about Microsoft’s Western Europe focused transformation priorities, programmes and actions. In addition to hands-on learning and pipeline generation, DSA is an opportunity to strengthen business relationships, and network across the Microsoft organisation and with other partners. Sponsoring partners will get to meet Microsoft Sellers, present at Industry Learning tracks and showcase their solutions and offerings at the Partner HUB.

This year’s conference will be held at the Rome Convention Center, from September 4 to September 6.

Applitools’ Co-founder and COO Moshe Milman will be presenting his talk in the Telco track on Thursday, Sept 5 — How Can Media Companies Release Fast While Maintaining a Great User Experience Across All Digital Channels by Leveraging Visual AI on Azure. 

We are proud to be a sponsor of Microsoft Digital Success Academy, where we will share the latest visual testing and monitoring techniques that support automation and improved visual UI testing through the Application Visual Management (AVM) approach!

If you are one of the 2000 lucky people that will attend this exclusive MSFT event, please stop by our booth — #12 in the Partner Hub — to learn more about the latest developments around end-to-end visual testing and monitoring techniques that can help your customers better support QA automation and continuous quality.

Additionally, make sure to check out the results from the 2019 State of Automated Visual Testing Report. Conducted as an independent survey of over 350 companies, the data outlines research findings for visual testing and quality, identifying key patterns that drive excellence in Application Visual Management.

We hope to see you there!

P.S. If you want to learn more about DevOps at Microsoft — this on-demand webinar is perfect for you: DevOps & Quality in The Era Of CI-CD: An Inside Look At How Microsoft Does It

The post Microsoft Digital Success Academy (DSA) 2019 – See You There! appeared first on Automated Visual Testing | Applitools.

Abel Wang, Principal Cloud Advocate for Microsoft Azure DevOps, spoke in an Applitools webinar recently. He called his webinar: DevOps and Quality in the Era of CI-CD: An Inside Look at How Microsoft Does It. Because he knows the workings of the team, Abel knows what it takes to operate Azure DevOps. In this webinar, Abel described why the build-test-release model was too slow, and he explained what drove changes in their development team enabled them to deliver software with both speed and quality. He explained that the combination of Azure DevOps plus visual testing results in better developer productivity, higher quality, even with frequent releases on a rapid cadence. And, Abel described how he came across Applitools as a way to provide visual testing for the Microsoft Azure DevOps development team, and why he likes Applitools so much.

I get sweaty palms thinking about some things that just freak me out. For example, my hands sweated profusely when I watched Free Solo, about climber Alex Hornnold who climbed El Capitan in Yosemite National Park without the benefit of a rope. I also get sweaty palms considering a change from a build-test-release model to a continuous integration – continuous deployment (CI-CD) model. So I appreciate hearing from someone who has done something successfully and is willing to share lessons they learned along the way.

Requirements Conflict With Conventional Wisdom

Conventional wisdom says that, for any project, you can have targets for scope, quality, and resources. Two targets remain to be fixed, but one must give way. When applied to build-test-release, this conventional wisdom means you’re either going to miss your scope, your date, or your quality target.

Companies that consider a shift to CI-CD are doing so hoping to break this conventional wisdom. Abel described the following as key issues that the Azure DevOps at Microsoft needed to address:

• Rapid Change – Customers demand new capabilities, and Azure DevOps wants to respond. The platform changes every three weeks, with bug fixes released in the interim.
• Quality – Customers expect existing features to continue to behave properly, and they expect new features not to impact existing behavior.
• Customer Choice – In today’s Microsoft, the company cannot dictate which tools customers can or should use. Customers want to choose their own tools and interface with other tools using standards.
• Reliability – All with no downtime

So, anyone in any CI-CD solution requires flexibility, quality, speed, and reliability. That clearly violates conventional wisdom. Microsoft maps out all the potential capabilities needed, and software tools to use, in a map like this:

Image courtesy Abel Wang, Microsoft

Clearly, Microsoft Azure DevOps expects customers to use any kind of tool for any purpose and have it work in their flow.

Why Waterfall Won’t Work

Next, Abel explained the old way of doing things at Microsoft, and why it just wouldn’t work for Azure DevOps.

Abel related his stories of development at Microsoft – specifically his motivations when he was a developer. His job, he recalled, was to develop code. Once it was done, checked in, and validated, he was done – and hopefully onto another project that was interesting and exciting. Testing was QA’s responsibility. Developers didn’t know much about the quality of any unit tests they wrote.

QA spent the bulk of their time writing and running functional tests. Small products could take months. Large products like a Windows release could take as long as a year. During the testing time, bugs were discovered and fixed, and the bug list burned down. Eventually, the release date would arrive, and the big issue for the product team was whether they thought customers could live with the remaining bugs.

The time scales

Development (Months) > QA (Months) > Release

cannot work for services like Agile DevOps. Microsoft needed a different workflow for developing software that brought speed and quality into the development process.

Microsoft’s Solution: Shift Left

Abel then relayed the five strategic changes that Microsoft employed to deliver speed and quality to their software development. They were:

• Make developers responsible for software quality
• Make developers feel the pain of their bad code
• Employ heavy use of Code Reviews / Pull Requests
• Move away from functional testing toward unit testing
• Feature Flags

This approach, commonly called “Shift Left” among developers, places the responsibility for software quality squarely on the developers.  Compared with the way things used to happen at Microsoft, the new approach for CI-CD required a radical shift for developers. And, as Abel said, most people were freaked out by their responsibilities.

Responsible for Quality

Making developers responsible for software quality meant that they had to think beyond their piece. Developers were required to consider how their code would be used in the real world. As a result, they started having to measure code coverage in their unit tests. Most of them had never considered coverage seriously. At first, they were frustrated by the extra work this entailed. Until they experienced the real pain of the code they had written.

Feel The Pain

What pain? Microsoft required each team to assign a single developer to be responsible for the team’s code during off hours, and the assignment rotated from person to person. If a production bug were found, the responsible engineer received a notification and had to join a bridge call – with all the executives for the business on the call.  And, while the executives were pushing for outcomes, it was up to the software engineer to fix the problem push the fix into production, and ensure that the root cause was identified and resolved the problem never, ever happened again.

Abel related from experience that, when you knew that the top people of the cloud business at Microsoft were on the call and demanding outcomes, that was both stressful and painful. The engineer who resolved that bug never wanted to be on a call like that again. And that pain made its way back to the team.

Use Code Reviews/Pull Requests

Once engineers had first-hand experience of the pain of an executive-level bridge call due to bad code, they all understood the need for quality. This drove the next behavior – code reviews and pull requests. Teams agreed that individuals could not simply push code into a build. Quality depended on the team doing reviews to ensure that the code would work across all the cases to which the team was developing.

Move To Better Unit Tests

The pain from bridge calls also drove another quality behavior in development – better unit tests. While the initial response to being responsible for the quality of their own code was frustration, developers changed their behavior. They started understanding the value of comprehensive test code to validate their work. And they began writing code with more comprehensive coverage. As a result, team code improved, and the unit tests could be run during code check-in.

These better unit tests also dropped the need for large functional test suites. These tests, which could take hours to run and even longer for root cause identification of failures, were needed for fewer and fewer test cases. In most situations, the unit tests validated functional behavior sufficiently so feature tests were unnecessary. Functional testing for behavior accounted for a huge chunk of the QA time in the build-test-release model. The increased coverage of unit testing decreased the amount of functional testing that was required.

Use Feature Flags

The last Shift Left process change involved feature flags. New features could be released to production flagged, so only certain users could see how they were behaving. As a result, new features could be tested in production! If the feature behaved, the flag could be changed and anyone might be able to use this feature. If errors were discovered, the feature could be removed or fixed in the next build without impacting paying customers.

Test for Real Eyeballs: Applitools

While the developer process changes with Shift Left resulted in faster delivery of features with quality, one big change was missing: visual errors. A small CSS change could have a big impact on the visual behavior of the application, and neither unit testing nor functional testing alone could spot this kind of error consistently.

In the past, Microsoft needed to run the functional tests in front of a group of manual testers. They could see the behavior of the application on-screen and determine whether or not the application behaved as expected. An army of manual testers had to run the tests and do comparisons to see how individual pages responded on individual browser/operating system combinations.

Abel relates that, in this quest for tools, Microsoft came across Applitools. As Abel said:

“There is a really cool technology from Applitools, it’s not even a Microsoft product, that really helps with this. It uses artificial intelligence to do visual testing. Basically, you write your automated UI test, (I write my test using Selenium). You can use Applitools to take pictures of your screen for you. Then they can do comparisons between what your baseline image should be and. what the images for your latest code check-in, and they can flag those differences for you.”

“Or if you want to keep those differences you can say you know what this is correct. The changes correct. Well let’s go ahead and set that as my new baseline moving forward. So you have the power to do that. And now what is so incredibly cool about this is you’re shifting even left. Even your testing is being shifted left – your manual or your automated UI testing has shifted left – where instead of needing human eyeballs all the time you can use Applitools to act as your human eyeballs. And only if there is a difference it will let you know and then you can now decide, ‘Is this a change you want or did somebody mess up my CSS file and now things look wonky? So it is super powerful and super useful.

Demonstrating Azure DevOps Plus Visual Testing with Applitools

Abel then went into a demo of his own environment.

“So what I want to do is show you just how easy it is to use Applitools and to integrate that into our Azure DevOps pipelines.”

Using Page Objects in Selenium

“I’m using Selenium to write my test. And when I write my automated UI test, I use a page object type of pattern where I create an object for every single page that my application has. In this page object, it’s just a simple object but it has all the actions that you can do on that particular screen. And it also has all the verifications that you can do on that screen. So for instance actions would be like, oh, well, you can click click on the exercise link or maybe I can enter in text into this specific text box. Those are all the actions that you can do on that screen.”

View the code on Gist.

“I also include the verification that you can do. So maybe I can enter this text, click on next, and then there should be a text box that pops up, or whatever. I need to verify that that text box shows up. I write tests that can actually show me that as well.

“When you write your test in this page object pattern, maintaining your code incredibly easily. You separate the test code from the page object. If your page changes, you just need to modify your page object.”

Reading Selenium Tests

“The other benefit is that my automated UI tests become incredibly easy to read as well. For instance, here’s my sample Healthy Food web app, and I’m going to go ahead and launch it. I pass in the browser, which is going to be Chrome, and then…

“Let’s go ahead and browse to the home page of the application. I pass in the URL. Then I verify my home page will be reached. I click on the Nutrition in the app link I remove all the donuts from the fields. Next, I create a new link. Now, I verify the page then read I had a donut in there I verify that the donut shows up on the table. So even if you’re not a coder you can look at this and be like: ‘Oh I know exactly what this test is doing.’ It just makes for a very very clean way to write your automated UI test.

“If we go ahead and jump into my launch code you can see this is where I actually do my selenium. So the first thing that I do is I check the browser and if I pass in the correct browser and Chrome I create a chrome driver. Then I launch the browser I set the size of the window and then the next thing that I do is I browse to my homepage. So if I go to browse to my homepage I pass into your URL and I just say “driver.navigate”. Go to the URL and I return to my home page and I pass in the driver.”

Adding Applitools

“Now I have these tests I want to add Applitools the ability to take pictures of my screens as I’m running through my tests so that I can compare our future tests with my base images to make sure nothing has changed or things haven’t turned wonky or weird with my UI from a visual perspective. It’s actually super easy to add Applitools into your tests.”

View the code on Gist.

“If you look at my test now, there’s a couple of things that I need to gather before I even run. Number one is I need to have a specific Applitools key. So I need to have that key if I want to run my Applitools type stuff in my build and release pipeline in Azure DevOps. I also grab a batch name and batch ID that automatically populates my environmental variables as well. Let me add that little chunk of code.

“Now let’s go into my test. Let’s go into this. I launch my web app. Next, I created chrome driver but now I grab the batch name in the batch ID. Then, I create an Eyes object which is part of the Applitools API which you can just get. This is the important code. I create my eyes object I call eyes.Open. I set Firefox to my window size. And here I set Chrome to the same window size as well. So the next thing I do is browse to my home page verify that the home page has been reached. Then I had this method called take a visual picture.”

View the code on Gist.

“When I take a visual picture it’s literally one line of code that I do I set eyes.CheckWindows and of course I pass in a tag that will show me all the data. Just helping organize stuff a little bit easier. So now what that does is on this particular page. Applitools will snap a screenshot and it can set it as my baseline. Or I can set it as manually set it as a baseline and then future tests, it will compare the pictures that it takes with the baseline to say has anything changed and if it has, is this a mistake or do you want to set your new changes as your new baseline. So super super super simple to do so once I add this check.

“If something looks wonky even though I’m on the right page like for instance the CSS got changed. Not a big deal because guess what Applitools will flag you and let you know. Super powerful super easy to use. The code changes that you make its almost nothing. So what does this look like? How do you set this up so it’s automated in your build and released pipelines? Well, let me go ahead and jump and show you what that looks like.”

 Azure DevOps Plus Visual Testing with Applitools

“Here’s my build pipeline and here’s my Applitools build and I’ll go ahead and edit it to show you guys what that looks like now. This build should look incredibly similar to the previous build because it’s literally the same thing but I added a couple of things. I added the Applitools build task and you can get this build task from the marketplace. You can add it to your marketplace and I’ve already installed it so I can now describe and drop it where I need to. Add it right before you compile your application. Then I run my unit test and then if everything looks good I go ahead and deploy my application into a C.I. environment. And now I run my automated UI test using selenium and that is literally all I have to do.”

“So we can come in here we can look to build summary. We see that 100 percent of our test is passed (hooray for that). That looks good. And if we look at the Applitools tab (you get a brand new Applitools tab now), it will actually show you all the pages that that were done. Everything looked good. Everything is passed. If you notice if I click on these you’ll be able to see screenshots of every time I took a picture.”

Resolving Visual Testing Differences with Applitools

“We’ll go ahead and jump into my summary and you’ll notice here 90.9% percent of my tests have passed. Two of them are in the “Other” category. Well, that’s weird. You’ll notice two of my automated UI tests are returning back “inconclusive”. If I look at Applitools (because that’s what my automated UI tests are running) I’ll notice that things are starting to look weird and Applitools flagged it for me. And if I jump in here and actually take a look at this you’ll notice that the table is no longer nicely formatted. Somebody must’ve messed up the CSS. Logically everything still works but visually it doesn’t look good. But I didn’t need a human to tell me that Applitools actually told me. So now I can either say, ‘This is the way I want it.’ or not. Thumbs up, it’s ok. Thumbs down, nope.”

Let’s go ahead and thumbs down. Right now. It will mark this as failed. Just like that. Now, future builds will be able to figure out what’s going on.

“This makes me so incredibly happy because it’s really powerful. It shifts automated UI testing to the left. And it makes our pipelines go faster and smoother. Super cool. Super, super useful. I am not an Applitools expert. Not even close to it. I’m just a code slinger. I ran into this toolset.  It was freaking amazing how easy it was to use and how useful.”

See The Full Webinar

The webinar is also covered in this blog post link.

Abel’s full code examples on GitHub.

The original slide deck on slideshare.com:

Find out more about Applitools

Setup a live demo with us, or if you’re the do-it-yourself type, sign up for a free Applitools account and follow one of our tutorials.

If you liked reading this, here are some more Applitools posts and webinars for you.

1. Visual UI Testing as an Aid to Functional Testing by Gil Tayar
2. How to Do Visual Regression Testing with Selenium by Dave Haeffner
3. Why Visual UI Testing and Agile are a Perfect Fit
4. The ROI of Visual Testing by Justin Rohrman

The post DevOps Plus Visual Testing: Abel Wang of Microsoft appeared first on Automated Visual Testing | Applitools.

Watch this on-demand session for an inside look at how Microsoft navigates DevOps in the era of CI-CD.

Abel Wang — Sr. DevOps expert @ Microsoft — discussed what DevOps is, why it’s important, and what products help MSFT solve DevOps challenges.
Abel also showed how Microsoft copes with, and solves, a major DevOps challenge: maintaining quality across its Dev/Test pipeline.

With 70% of companies citing CI-CD as a top strategic initiative, we no longer question the need for DevOps — but implementing a successful DevOps practice and maintaining app quality remain challenging for companies of all sizes.

At Microsoft, DevOps means something specific: DevOps is the union of people, processes and products enabling the continuous delivery of value to end users.
This union isn’t easy.
Aligning these three pillars takes careful planning. Making sure the right culture, the right software development processes, and the right products and tools are in place enables this comprehensive DevOps mindset.

Among the topics covered: 

• 3 Pillars of DevOps — as defined by Microsoft
• How MSFT organized its team for DevOps success
• DevOps best practices — recommended by the experts @ MSFT
• How to maintain quality while continuously delivering apps

Abel’s Slide Deck:

Abel’s Code Examples on GitHub

Full webinar recording:

Additional Resources and Recommended Reading:

1. Putting Quality First: Evolution of Dev/Test and DevOps in the Era of Continuous Delivery — webinar with Joe Colantonio and Adam Auerbach
2. How to Update Jenkins Build Status in GitHub Pull Requests — Step-by-Step Tutorial by Sophie Tagar
3. How to Test Your GitHub Pull Requests so You Don’t Break Your UI – post by Al Sargent
4. Integrate Applitools Eyes Test Results Into Slack — Step-by-Step Tutorial by Eran Barlev
5. How to Improve Customer Experience with Visual AI — interview with James Lamberti, Applitools CMO
6. Applitools for DevOps – Learn how Applitools can help you continuously test and monitor the UI of your app as part of your CI-CD process: monitor all the critical visual aspects of your digital business to be more confident, and frequent with your releases.
7. GitHub and Visual Studio Team Services Integrations – now in Applitools Eyes
8. Release Apps with Flawless UI: Open your Free Applitools Account, and start visual testing today.
9. Test Automation U — the most-talked-about test automation initiative of the year: online education platform led by Angie Jones, offering free test automation courses by industry leaders. Sign up and start showing off your test automation certificates and badges!

The post DevOps & Quality in The Era Of CI-CD: An Inside Look At How Microsoft Does It appeared first on Automated Visual Testing | Applitools.

Applitools 10.2 is now generally available, and we’d like to let you know about the support we’ve added for GitHub and Microsoft Visual Studio Team Services (VSTS). These enhancements let you incorporate visual testing earlier in your release process.

GitHub Integration

Our new GitHub integration prevents visual issues from merging into your production software pipeline. Features include:

• Integration of Applitools visual tests with GitHub pull & push request workflow.
  • See visual tests results & merge status as part of the GitHub pull request panel (see screenshot below).
  • Resolve any merge conflicts between branches.
  • Ensure that only code with resolved issues merges in GitHub code.  

• Applitools baseline branching mapped to GitHub codebase branching
  • Applitools tests use and save baselines in branches using a name derived from Git branches.
  • Applitools batches together all visual tests in a specific CI build with a batch name that includes Git commit information. This provides more insight on issues detected.
• Compare baseline Applitools visual test results across branches as part of the GitHub merge process.
  • Merges driven by a pull request trigger an Applitools branch merge to keep alignment between Applitools visual test baselines and your codebase in GitHub.

Getting started with GitHub is very simple, with our integration available right from the GitHub store.

To be clear, our GitHub integration is only available for Applitools Public Cloud. We’re working to bring GitHub integration to our Private Cloud and On-Premise customers soon.

Visual Studio Team Services Integration

We also released a plugin for integrating with Microsoft Visual Studio Team Services (VSTS) build processes. Our Applitools plugin for Microsoft VSTS expands our enterprise CI/ CD integration beyond our existing Jenkins and JetBrains TeamCity integrations. Here’s a view of the plugin installed into VSTS:

Next Steps

Learn more in the documentation for our GitHub Integration and our VSTS Integration, request a demo, or sign up for a free Applitools account.

The post GitHub and Visual Studio Team Services Integrations now in Applitools Eyes appeared first on Automated Visual Testing | Applitools.

“Applitools helps Visual Studio developers solve real, challenging problems for testing with a unique and innovative solution,” said Mitra Azizirad, General Manager, Developer Tools Marketing and Sales, Microsoft.

“Applitools Eyes, integrated with Visual Studio 2013, enhances Coded UI and adds automated visual testing for mobile and web apps, while also tying into our application lifecycle management tools.”

Tel Aviv, Israel, December 10th, 2013 – Applitools, innovators in Automated Visual Testing, announces being a Microsoft Visual Studio 2013 partner with Applitools Eyes, the first Cloud Service for Automated Visual Testing of Web, Mobile and Desktop applications, based on proprietary Cognitive Vision technology. 

“In today’s world, UI is the face of any application and a key to successful user experience (UX)” says Gil Sever, Co-Founder and CEO of Applitools. “The tremendous evolution in modern app environments such as multi-device, Agile, Continuous Deployment and more, is posing new challenges to developers and testers which are not being answered by any of the existing functional test automation tools. Applitools Eyes enables app developers and testers to bridge that gap and automate this process. We’ve presented Applitools Eyes to testers, developers and industry experts and received great feedback. “

Applitools Eyes can add value to Visual Studio users in three ways: First, it easily integrates with CodedUI and immediately enhances it with a unique and unprecedented Automated Visual Testing capability.

Second, once integrated with Coded UI, Applitools Eyes can be run and managed by Microsoft Test Manager (MTM) along with all the other functional tests.

Third, when Applitools Eyes detects visual and UI problems in the Application under Test (AUT), it returns an error notification to Microsoft TFS and a link to the visual log of the failure, which provides the user an immediate, exact and easy-to-understand description of the root cause of the problem. Since Applitools is a cloud service, it can easily integrate with the Cloud version of Microsoft TFS (Microsoft Team Foundation Service).

Applitools Eyes includes SDKs for CodedUI and Selenium .NET, which allows developers and testers that use Visual Studio, to use Applitools Eyes very easily. Applitools Eyes is compatible with Microsoft Visual Studio 2013 and easily integrates with MTM and TFS.

“Applitools helps Visual Studio developers solve real, challenging problems for testing with a unique and innovative solution,” said Mitra Azizirad, General Manager, Developer Tools Marketing and Sales, Microsoft. “Applitools Eyes, integrated with Visual Studio 2013, enhances Coded UI and adds automated visual testing for mobile and web apps, while also tying into our application lifecycle management tools.”

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Applitools Announces Being a Microsoft Visual Studio 2013 Partner appeared first on Automated Visual Testing | Applitools.