Let me tell you a story. It’s one I haven’t told before. But to do it, let’s first get acquainted.

Hi – I’m Dave Piacente. You may know me from a past life when I went by the name Dave Haeffner and my past works with Selenium. I’m the new DevRel and Head of Community at Applitools—Andy’s moved on to a tremendous bucket-list job opportunity elsewhere, and we wish him all the best! I’ve been working closely with him behind the scenes to learn the ropes to help make this a smooth transition and to ensure that all of the great work he’s done and the community he’s grown will continue to flourish. And to loosely paraphrase Shakespeare – A DevRel (or a Dave) by any other name would be just as sweet.

Now, about that story…

I used to be known for a thing – “Selenium Dave” as they would say. I worked hard to earn that rep. I had one aim, to be helpful. I was trying to solve a problem that vexed me early on in my career in test automation (circa 2009) when open-source test automation and grid providers were on a meteoric rise. The lack of clear and concise guidance on how to get started and grow into a mature test automation practice was profound. But the fundamentals weren’t that challenging to master (once you knew what they were), and the number of people gnashing their teeth as they white-knuckled their way through it was eye-popping.

So, back in 2011, after working in the trenches at a company as an SDET (back before that job title was a thing), I left to start out on my own with a mission to help make test automation simpler. It started simply enough with consulting. But then the dominos began to fall when I started organizing a local test automation meetup.

While running the meetup I realized I kept getting asked the same questions and offering the same answers, so I started jotting them down and putting them into blog posts which later became a weekly tip newsletter (Elemental Selenium, which eventually grew to a readership of 30,000 testers). Organically, that grew into enough content (and confidence) to write a book, The Selenium Guidebook.

I then stepped out of meetup organization and into organizing the Selenium conference, where I became the conference chair from 2014 to 2017. My work on the conference opened the door for me to become part of the Selenium core team. From there it was a hop-skip-and-a-jump to working full-time as a contributor on Selenium IDE at Applitools.

Underpinning all of this, I was doing public speaking at meetups and conferences around the world (starting with my first conference talk back in 2010). I felt like I had summited the mountain—I was in the best possible position to be the most helpful. And I truly felt like I was making a difference in the industry.

But then I took a hard right turn and stopped doing it all. I felt like I had accomplished what I’d set out to do – I had helped make testing simpler (at least for people using Selenium). So I stepped down from the Selenium project, I stopped organizing the Selenium conference, I stopped doing public speaking, I sold my content business (e.g., the newsletter & book) to a third party, and I even changed my last name (from Haeffner to Piacente – although for reasons unrelated to my work). By all marks, I had closed that chapter of my life and was happily focusing on being a full-time Software Developer in the R&D team at Applitools.

While I was doing that, the test automation space continued to grow and evolve as I watched from the sidelines. Seemingly every enterprise was now shifting left (not just the more progressive ones), alternative open-source test automation frameworks to Selenium continued to gain ground in adoption, some new-and-noteworthy entrants started popping up, and the myriad of companies selling their wares in test automation seemed to grow exponentially. And then, Generative AI waltzed into the public domain like the Kool-Aid man busting through a wall. “Oh yeah!”

I started to realize that the initial problem I had strived to make a dent in—making testing simpler—was a moving target. Some things are far simpler now than when I started out, but some are more complex. There are new problems constantly emerging, and the ground underneath our feet is shifting.

So perhaps my work is not done. Perhaps there is more that I can do to help make test automation simpler. To return to public speaking and content creation. To return to being helpful. But this time, with the full weight of a company behind me, instead of as just as a one-man show.

I’m thrilled to be back, and I’m excited for what’s to come!

The post Welcome Back, Selenium Dave! appeared first on Automated Visual Testing | Applitools.

Everyone has their own opinions. And some of them? Preposterous. Others? Downright controversial! Like, for instance, thinking that The Bourne Legacy is the best of all the Bourne movies (change my mind). Or that vim is better than emacs… and VSCode (!). You get the idea, and this is nothing new.

But what never gets old is when you can find a group of people where you can connect regardless of your opinions and feel like you belong. To find that group of people who can take you for you are (regardless of your poor taste in movies or questionable choice of text editor), riff on it, and (lovingly) rib you a bit for it. To do this in all facets of life is important, but most recently, I managed to find this group of people in my work life – having recently joined Applitools as a Software Developer. And ironically, this isn’t my first time working here.

In the not-too-distant-past I took some time away from my career to focus on my family. My wife was looking to head back to work after a few years away to focus on raising our young children. I was looking to take a break from work and endeavor on a different kind of challenge (family circus conductor). Fast forward to the end of what I affectionately refer to now as my extended sabbatical, things are different. My kids are older now and in preschool, my wife is back working, and I’m home twiddling my thumbs wondering “What to do?”

So I explored a few options.

Back into entrepreneurship? Sure, why not? So I looked into starting a new business. But man, that’s a lot of work! Last time I did that was in 2011. I did it by myself then and it was very hard. The conclusion? That’s a young man’s game. Nope, next.

Why not partner with someone instead of going it alone? Okay, sure. So I joined a friend’s startup as a technical co-founder. But that didn’t feel like the right fit either. So maybe freelance software developer? Done. That started out okay, but after a few months I realized it was lonely and not challenging me in the ways I was looking to grow. Hmm.

At the end of it all, a question crystallized for me, “What do I want to do and who do I want to do it with?” Me, a vim user with fantastic taste in films. Where I ended up was a headspace eerily similar to where I was in 2018.

Back then I decided that I wanted to make a go of being a full time software developer. To focus on the process of making high quality, high value software. But here was the rub. While I had experience working with a handful of programming languages (at least superficially through my work in software testing), I didn’t have a “proper” background. I don’t have a degree in computer science (I have a degree in network engineering, thank you very much), I’ve never worked as a developer for a company, and hilariously I failed my intro to programming course at university (I did much better the second time though!). But through my work in software testing I was able to parlay that into a position as a software developer working at Applitools, which turned out to be a life-changing experience for me. I got to work with immensely smart and talented people who welcomed me warmly, helped bring me along, and challenged me in ways that supercharged my growth (shoutout to Doron, Tomer, Gil, Amit, and Adam!). And it didn’t hurt that I got to work on fascinating, multi-faceted technical problems that forced me to grow my problem solving skills every day.

I remembered all of this fondly when searching for an answer to my question – “What do I want to do and who do I want to do it with?” Not only did I realize I wanted to continue my journey of software craftsmanship but I also wanted to go back to working alongside great engineers in a collegial environment. With people who both accept me as I am and challenge me to be better. To be in a place where I’m fed an endless supply of technical problems which are fascinating to a software geek like me.

On the tin, this is Applitools. And not for nothing, it also doesn’t hurt that they are building the most innovative stuff in the software testing space. I say this non-hyperbolically with over a decade in this industry (“I’ve seen things you people wouldn’t believe”).

So I was floored when I messaged my old manager to reconnect and tell him what I was thinking. Because very quickly this started a chain reaction of conversations which led to me ultimately answering the question “When can you start?”. Before I knew it, my start date was upon me. And you know what? I was welcomed back just as warmly as when I joined the first time. Now I’m well on the other side of my start date, back in the trenches, working alongside my fellow colleagues. And I gotta say, it’s great to be back!

Interested in joining? Come for the people and technical problems, but stay for the innovation that’s shaking up software testing (and maybe a movie recommendation or two :-)). Take a look at job openings here: http://www.applitools.com/careers

The post Why I Joined Applitools (again): Dave Haeffner appeared first on Automated Visual Testing | Applitools.

Dave Haeffner has had a varied career that began with a degree in networking engineering. He worked various roles in IT operations, eventually stumbling into QA and test automation without a background in programming. This started him on a path that ultimately lead him to author ‘The Selenium Guidebook‘. Along the way, he’s helped various companies implement automated acceptance testing (The Motley Fool, ManTech International, Sittercity, and Animoto), spoken at numerous conferences and meetups around the world about how to use Selenium successfully, and is an active contributor to the open source Selenium project.

In June 2018 he moved to Tel Aviv from the US to join Applitools as the second member of our open-source Selenium IDE team, working closely with Tomer Steinfeld. We sat down with Dave to talk us through his career path and the work he currently does to improve Selenium IDE.

Hi Dave! Thank for joining us! To begin with, could you tell us a little about your background and how you ended up becoming a Selenium expert?

I actually went to college for network engineering at a small state school in upstate New York, which is a business school. I chose this program because, at the time, I was really into computer networking and gaming. When I discovered that there was a degree that meant I could do the things I love for a living I was sold!

After qualifying, I began working as a network engineer in a state hospital. It was a very specific role. I quickly realized I needed more opportunities for career growth. With this in mind, I moved to DC to work as a support engineer for a boutique consulting firm. Eventually, I got burnt out on the work and became intrigued by the startup culture that was beginning to flourish in the tech industry.

A friend of mine worked at The Motley Fool. The concept of a flat organizational structure, working with great people and learning more about software testing drew me in. I joined The Motley Fool as part of their desktop support team and used this platform to learn more about their software development team. After a couple of years in that position, there was an opportunity to transition into a role in QA. I wasn’t really sure what that entailed, but I was up for a new challenge.

How Was Your Transition to QA?

As it turns out, test automation was a hard thing in 2009. My new team had invested quite a lot of effort into test scripts with Selenium IDE and were having loads of issues with maintaining them. Like me, none of my teammates had a background in programming. This is really where my software testing story began.

I was fortunate to be in complete ignorance of how much I didn’t know. I dug in and started to look for ways to make the Selenium IDE scripts more maintainable and reliable. The wisdom of the crowd suggested that you needed to export your tests to a programming language and mold this code into something reasonable (e.g., a test harness). Unsure if we were on the right track, we gave it a shot. It seemed to help, but it was difficult as we were all fairly new to programming. That same year I had the opportunity to attend the annual Agile conference. While there, I got to meet the creator of Selenium, Jason Huggins. I told him about our trials and tribulations with Selenium IDE and code export. He said ‘It’s really hard, but you’re on the right track, kid.”

Filled with props from Selenium’s creator, I returned from the conference eager to keep going. Along with the help of some of my developer colleagues, we were able to make significant improvements in our automated testing practice. The following year, I returned to Agile as a speaker to share our story and lessons learned. As a result, Selenium became such a big part of my life that it afforded me the opportunity to branch out on my own and move into the world of consulting.

What was it like branching out on your own into consulting?

I’d always wanted to be an entrepreneur and the consulting thing naturally grew from my interest in Selenium. Working with a group of customers I gained insight into the limitations that teams encountered when trying to adopt practices using Selenium and how difficult it can be for teams to use test automation. I quickly realized that a lot of the problems around the tooling were the same – the same impediments, the same questions, even in the same order. It was like a Power Law! So I began writing them down and decided to organize a local Meetup to connect and learn from other practitioners.

I kept getting a lot of the same questions over email about Selenium and realized that it would be a lot easier to answer the question once, publish the solution online, and refer people to it. This is how I started my weekly Selenium tip newsletter (Elemental Selenium). Eventually, my work with Selenium started to become a known quantity, and my book (The Selenium Guidebook) evolved from this.

What convinced you to move your family to Tel Aviv and work for Applitools?

Back when I was applying for college, I had the opportunity to look at computer science. At that point, after being homeschooled I wasn’t confident in my math abilities, which put me off applying. I guess, I always wondered what it would be like. So, while I was consulting I tried to fill that void through self-teaching – starting with some of the fundamental programming books (e.g., The Gang of Four book, Working Effectively with Legacy Code, and Refactoring) and applying them to projects as I went along.

To be honest, I was hesitant to work with a company. After being self-employed for the better part of a decade I thought it would be a tough transition. Also, I wasn’t sure if a company would be open to hiring someone who is an expert in one area but mediocre in others. But Adam Carmi (CTO at Applitools) saw an opportunity for my expertise that would also allow me to work on my software development skills. Applitools is making an impressive investment in the IDE, with two people to working full-time on its development (me & Tomer Steinfeld). It was a unique and rare opportunity that I couldn’t pass up. Plus, I’d always wanted to live in another country.

Right now, I’m working with Doron Zavelevsky and Tomer Steinfeld. Their depth of knowledge is impressive, as is their willingness for knowledge sharing. As I work on Selenium IDE, I often reflect back to my early days with the tool and think — I’ve come full circle.

Sounds like it’s working out to be a great match! Could you tell us a little about your role?

Working in open source is great, but it’s not without challenges. We have a lot of constraints, both technical limitations and a large backlog of development and feature requests. Selenium IDE is built as a web extension which also presents some unique challenges. As does writing record and playback functionality in the browser (hello edge cases!). But it’s a lot of fun, and we learn something new every day.

You Must Have A Lot On Your Plate!

Absolutely, but that’s what makes it interesting. Selenium IDE has a history of neglect. It’s been good enough for some, but terrible for others. The amount of interest to push the tool forward often peters out. Here, it’s our job to make it into a robust tool that helps people with end-to-end testing in the browser. We’re working to make the tool more reliable, more powerful, and more intuitive. Ultimately my goal in this role is to remove the stigma attached to Selenium IDE and record-and-playback. I continue to learn good software development practices along the way (and also improve my Hebrew!).

What Else Do You Want Readers to Know About Applitools?

Our engineering team is hiring! Take a look at our job openings and get in contact to find out more: www.applitools.com/careers

The post Why I joined Applitools: Dave Haeffner appeared first on Automated Visual Testing | Applitools.

Watch this webinar, where Selenium Guru Dave Haeffner and Software Developer Tomer Steinfeld, showcase the New Selenium IDE.

In this webinar, you’ll learn how the newly re-factored Selenium IDE is guaranteed to augment your testing efforts, regardless of your team’s automation “maturity” level, or your personal technical knowledge or experience.

Watch this session with Dave Haeffner and Tomer Steinfeld – full-time maintainers of Selenium IDE — and learn: 

• Why codeless solutions are worth paying attention to
• 10 traits of a great codeless testing tool
• Key features in the New Selenium IDE:
  • Reliable test playback with “locator fallback”
  • Enhance tests with control flow logic (e.g. if, else if, else, while, do, while, times)
  • Built-in debugger for diagnosing issues in your tests
  • Turn-key parallel, cross-browser execution that can be used on CI
  • Add/modify commands in the IDE, or integrate with a third-party provider through plugins
• Roadmap – what’s in store for the New Selenium IDE

Dave also shared how he went from a record-and-playback naysayer to the Selenium IDE maintainer.

In addition, this session includes 20 minutes of live Q-and-A with Dave and Tomer.

Full slide-deck:

Full webinar recording:

Demos (from the session):

Parallel execution with the Runner (slide 13 in the slide deck):

Reusable Test Case (slide 17 in the slide deck):

Additional Materials and Recommended Reading:

“10 Features Every Codeless Test Automation Tool Should Offer” — article by Angie Jones (as mentioned in the session by Dave)

Selenium IDE Page – on the Selenium Project website

Selenium Conf London – October 7-8, 2019, London UK – Join 600 test automation pros and an amazing lineup of speakers, including: Richard Bradshaw, Ash Coleman, and more. CFP and Early Bird Ticket Sales open on Feb 27!

Selenium IDE: The Next Generation — webinar with the Godfather of Selenium: Simon Stewart, inventor of Selenium WebDriver.

Release Apps with Flawless UI: Open your Free Applitools Account, and start visual testing today.

Test Automation U — the most-talked-about test automation initiative of the year: online education platform led by Angie Jones, offering free test automation courses by industry leaders. Sign up and start showing off your test automation badges!

— HAPPY TESTING —

The post The New Selenium IDE: See How It Can Turbo-charge Your Testing Efforts appeared first on Automated Visual Testing | Applitools.

Test Automation thought leaders gathered for a round-table discussion about the upcoming trends, best practices, tools, and ideas that will shape your Dev/Test environment in 2019.

Joe Colantonio hosted this all-star expert panel, including: Angie Jones, Dave Haeffner, and Gil Tayar – as they shared their thoughts and insights on the hottest topics in test automation and software quality, including:

• Shift left: how to get developers involved and invested in the testing process
• Selenium IDE: ghosts of the past, present, and future
• Avoiding common pitfalls: how to be successful with test automation
• Never bet against it: JavaScript’s role in test automation
• New automation frameworks Cypress and Puppeteer: the good, the bad and the ugly
• Keep calm and record-and-playback

Listen to the recording:

Slide Deck:

— HAPPY TESTING — 

The post Test Automation in 2019: Glimpse of the Near Future From the Experts Who Shape It appeared first on Automated Visual Testing | Applitools.

The Problem

There is a stark misconception in the industry when it comes to visual testing. Namely that it is hard to automate. Even noted automaters in the field share this misconception, which can be seen by this quote.

“Visual testing, something that is very difficult to automate.” – Richard Bradshaw (@FriendlyTester) [source]

As a result, people tend to turn a blind eye to it and miss out on a tremendous amount of value from something that is getting easier to implement every day. 

A Visual Regression Testing Primer

Visual Testing (a.k.a. visual checking or visual regression testing or visual UI testing) is the act of verifying that an application’s graphical user interface (GUI) appears correctly to its users. The goal of the activity is to find visual bugs (e.g., font, layout, and rendering issues) so they can be fixed before the end-user sees them. Additionally, visual testing can be used to verify content on a page. This is ideal for sites that have graphical functionality (e.g., charts, dashboards, etc.) since verification with traditional automated functional testing tools can be very challenging.

Given the number of variables (e.g., web browsers, operating systems, screen resolutions, responsive design, internationalization, etc.) the nature of visual testing can be complex. But with existing open source and commercial solutions, this complexity is manageable, making it easier to automate than it once was. And the payoff is well worth the effort.

For example, a single automated visual test will look at a page and assert that every element on it has rendered correctly. Effectively checking hundreds of things and telling you if any of them are out of place. This will occur every time the test is run, and it can be scaled to each browser, operating system, and screen resolution you care about.

Put another way, one automated visual test is worth hundreds of assertions. And if done in the service of an iterative development workflow, then you’re one giant leap closer to Continuous Delivery.

A Solution

By using an existing solution, you can get up and running with automated visual testing quickly. Here’s a list of what’s available (sorted alphabetically by programming language):

NOTE: You may be wondering why Sikuli isn’t on this list. That’s because Sikuli isn’t well suited for automated visual testing. It’s better suited for automated functional testing — specifically for hard to automate user interfaces.

Each of these tools follows some variation of the following workflow:

1. Drive the application under test (AUT) and take a screenshot
2. Compare the screenshot with an initial “baseline” image
3. Report the differences
4. Update the baseline as needed

Let’s dig in with an example.

An Example

We’ll use WebdriverCSS, which works with Selenium WebDriver. Specifically, the NodeJS bindings and the Selenium Standalone Server (which you can download the latest version of here).

Our application under test is a page on the-internet which has a menu bar with a button that will render in a slightly different location (e.g., 25 pixels in either direction) every time the page loads.

After starting the Selenium Standalone Server (e.g., java -jar selenium-server-standalone-2.44.0.jar from the command-line) we’ll need create a new file. In it we’ll need to require the necessary libraries (e.g., assert for our assertions, webdriverio to drive the browser, and webdrivercss to handle the visual testing) and configure Selenium to connect to the standalone server (which is handled with desiredCapabilities).

# filename: shifting_content.js

var assert = require('assert');
var driver = require('webdriverio').remote({
  desiredCapabilities: {
    browserName: 'chrome'
  }
});
require('webdrivercss').init(driver, { updateBaseline: true });

This will provide us with a driver object that we can use to interact with the browser.

This object is a normal webdriverio instance with one enhancement — the addition of the .webdrivercss command. This command provides the ability to specify which parts of our application we want to perform visual testing on.

For this example, let’s keep things simple and focus on just the page body.

driver
  .init()
  .url('http://the-internet.herokuapp.com/shifting_content?mode=random')
  .webdrivercss('body', {
    name: 'body',
    elem: 'body'
  }, function(err,res) {
      assert.ifError(err);
      assert.ok(res.body[0].isWithinMisMatchTolerance);
    })
  .end();

After specifying our focus element we want to check to see that no visual changes occurred since the last time we ran this test. This is handled with an assert, the focus element (e.g., res.body[0]), and the isWithinMisMatchTolerance command.

The mismatch tolerance is configurable (on a scale of 0 to 100) but is defaulted to 0.5. There are other configuration options with sensible defaults as well (e.g., the folder where screenshots are stored) but you can ignore them for now.

Each time this script is run WebdriverCSS will take a screenshot. The initial shot will be used as a baseline for future comparisons. Then on each subsequent run WebDriverCSS will check to see if the new screenshot is within the mismatch tolerance from the baseline image. If it’s not, then the script will fail.

If the script fails enough times, then the baseline image can be automatically updated. But only if we tell WebdriverCSS to do it (which we already did in our initialization of it).

require('webdrivercss').init(driver, { updateBaseline: true });

Expected Behavior

If we save this file and run it (e.g., node shifting_content.js from the command-line) here is what will happen:

• Open the browser
• Navigate to the page
• Take a screenshot of the page
• Compare the screenshot against the baseline
• Assert that the targeted area of the page is within the match tolerance
• Fail and render a diff image on local disk

In Conclusion…

Hopefully this write-up has gotten you started off on the right foot with automated visual testing. There’s still more to consider when it comes to scaling your automated visual testing. But don’t fret, we’ll cover that in the following write-ups.

Until then, feel free to check out UIRegression. It’s another great online resource for learning about visual testing.

Read Dave’s next post in this series: How to Handle False Positives in Visual Testing

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post How to Do Visual Regression Testing with Selenium appeared first on Automated Visual Testing | Applitools.

Interested in learning what are the key skills expected from software testing professionals in modern development? Want to hear first-hand where is Selenium going for the future of test automation? Then you must listen to this on-demand session!

A panel of four experts in Test Automation and free and open-source software shared with us their opinions on the state of testing, the hottest trends in test automation, and their vision on the future of Selenium for test automation.

Dave Haeffner, Ashley Hunsberger, Karen Sandler and Joe Colantonio joined forces to discuss emerging trends that will shape your testing environment in the months and years to come. Their discussion also includes a few key takeaways and highlights from the last Selenium Conference that took place last month in Austin Texas.

Watch the on-demand recording of the State of Selenium and Test Automation webinar now:

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account. Happy testing!

The post State of Selenium and Test Automation: 4 Leading Experts Share Their Vision of Things to Come appeared first on Automated Visual Testing | Applitools.

Perhaps you’ve dabbled with Page Objects, Wait Strategies (aka Implicit and/or Explicit Waits), and you feel confident about your locators. Your test code might be in pretty good shape — able to work reliably as time marches on and the application under test continues to evolve and your testing needs continue to grow with it.

But how do you know your test code is in good shape? It’s not like there is a quantitative way to measure this.

Or is there? 

Watch this advanced session Dave Haeffner’s Proven Method to Grading the Quality of Selenium Tests as Selenium expert Dave Haeffner steps through the core tenets of good test and page object design, locators, and a repeatable and quantitative approach for assessing your test code. When you’re done, you’ll be able to see how your tests and page objects stack up, and what changes are needed to help them stand the test of time.

Dave’s slide-deck can be found here:

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Dave Haeffner’s Proven Method to Grading the Quality of Selenium Tests appeared first on Automated Visual Testing | Applitools.

It was an awesome webinar – and now you can watch it on-demand, right here!

What is Selenium? Why should you use it? And how do you use it successfully?
In this webinar, Automation expert Dave Haeffner answered these questions as he steps through the why, how, and what of Selenium.

Dave also discussed how to start from nothing and build out a well factored, maintainable, resilient, fast and scalable set of tests.
These tests will not only work well, but across all of the browsers you care about, while exercising relevant functionality that matters to your business.

Watch this webinar, and learn how to: 

• Decompose an existing web application to identify what to test
• Pick the best language for you and your team
• Identify which browsers to test with & setup Selenium to work with them
• Write maintainable and reusable Selenium tests that will be cross-browser compatible and performant
• Dramatically improve your test coverage with automated visual testing
• Build an integrated feedback loop to automate test runs and find issues fast

Watch it below:

And find the slides here:

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Mastering Test Automation: How to Use Selenium Successfully – presented by Dave Haeffner appeared first on Automated Visual Testing | Applitools.

Our friend Dave Haeffner (Test Automation expert extraordinaire and author of The Selenium Guidebook) has compiled the best Selenium resources into one comprehensive list, which covers: 

• Documentation & Tips
• Prominent Blogs
• Books
• Meetups
• Conferences
• Videos
• Mailing Lists
• Forums
• Issues
• (and last but not least), The Selenium IRC Chat Channel

Get the full list here, on Elemental Selenium (Dave’s Selenium Tip website). And you can get weekly tips like this sent directly to you by signing up for Dave’s Selenium Tip Newsletter here.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post 24 Top Selenium Resources appeared first on Automated Visual Testing | Applitools.

Must-see webinar for test automation professionals, presented by Automation expert Dave Haeffner – now available online, on-demand!

Learn how to build simple and powerful automated cross-browser tests, covering visual testing and functional regressions, and configured to run automatically through the use of a Continuous Integration (CI) server.

Watch this step-by-step webinar, presented by Test Automation expert Dave Haeffner (author of Elemental Selenium and The Selenium Guidebook), and learn how to: 

• Get started off on the right foot with Selenium by performing hundreds of assertions with just a few lines of code
• Run your automated web tests on any browser and operating system combination
• Automate your test runs with a Continuous Integration server

Watch it now:

Dave’s slides can be found here: 

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Awesome Test Automation Made Simple – presented by Dave Haeffner appeared first on Automated Visual Testing | Applitools.

Automating Your Test Runs with Continuous Integration 

 

You’ll probably get a lot of mileage out of your automated tests if you run things from your computer, look at the results, and tell people when there are issues in the application. But that only helps you solve part of the problem.

The real goal in test automation is to find issues reliably, quickly, and automatically – and ideally, in sync with the development workflow you’re a part of.

To do that we need to use a Continuous Integration server.

A Continuous Integration Server Primer

A Continuous Integration server (a.k.a. CI) is responsible for merging code that is actively being developed into a central place (e.g., “trunk” or “master”) frequently (e.g., several times a day, or on every code commit, etc.) to find issues early so they can be addressed quickly — all for the sake of releasing working software in a timely fashion.

With CI, we can automate our test runs so they can happen as part of the development workflow. The lion’s share of tests that are typically run on a CI Server are unit (and potentially integration) tests. But we can very easily add in our recently written Selenium tests.

There are numerous CI Servers available for use today, most notably:

• Bamboo
• Jenkins
• Solano Labs
• TravisCI
• etc.

Let’s step through an example of using Jenkins on CloudBees.

An Example

Jenkins is a fully functional, widely adopted, and open-source CI adn CD (Contibuous Delivery) server. Its a great candidate for us to step through. And DEV@cloud is an enterprise-grade hosted Jenkins service offered by CloudBees, the enterprise Jenkins company. It takes the infrastructure overhead out of the equation for us.

1. Quick Setup

We’ll first need to create a free trial account, which we can do here.

Once logged in we can click on Get Started with Builds from the account page. This will take us to our Jenkins server. We can also get to the server by visiting http://your-username.ci.cloudbees.com. Give it a minute to provision, when it’s done, you’ll be presented with a welcome screen.

NOTE: Before moving on, click the ENABLE AUTO-REFRESH link at the top right-hand side of the page. Otherwise you’ll need to manually refresh the page to see results (e.g., when running a job and waiting for results to appear).

2. Create A Job

Now that Jenkins is loaded, let’s create a Job and configure it to run our tests.

1. Click New Item from the top-left of the Dashboard
2. Give it a descriptive name (e.g., Login Tests IE8)
3. Select Freestyle project
4. Click OK

This will load a configuration screen for the Jenkins job.

 

3. Pull In Your Test Code

Ideally your test will live in a version control system (like Git). There are many benefits to doing this, but the immediate one is that you can configure your job (under Source Code Management) to pull in the test code from the version control repository and run it.

1. Scroll down to the Source Code Management section
2. Select the Git option
3. Input the Repository URL (e.g., https://github.com/tourdedave/getting-started-blog-series.git)

Now we’re ready to tell the Jenkins Job how to run our tests.

4. Add Build Execution Commands

1. Scroll down to the Build section
2. Click on Add Build Step and select Execute Shell

In the Command input box, add the following commands:

export SAUCE_USERNAME="your-sauce-username"
export SAUCE_ACCESS_KEY="your-sauce-access-key"
export APPLITOOLS_API_KEY="your-applitools-api-key"
gem install bundler
bundle install
bundle exec rspec

Since our tests have never run on this server we need to include the installation and running of the bundler gem (gem install bundler and bundle install) to download and install the libraries (a.k.a. gems) used in our test suite. And we also need to specify our credentials for Sauce Labs and Applitools Eyes (unless you decided to hard-code these values in your test already – if so, then you don’t need to specify them here).

5. Run Tests & View The Results

Now we’re ready to save, run our tests, and view the job result.

• Click Save
• Click Build Now from the left-hand side of the screen

When the build completes, the result will be listed on the job’s home screen under Build History.

You can drill into the job to see what was happening behind the scenes. To do that click on the build from Build History and select Console Output (from the left navigation). This output will be your best bet in tracking down an unexpected result.

In this case, we can see that there was a failure. If we follow the URLs provided, we can see a video replay of the test in Sauce Labs (link) and a diff image in Applitools Eyes.

The culprit for the failure here wasn’t a failure of functionality, but a visual defect with the image on the Login button.

A Small Bit of Cleanup

Before we can call our setup complete, we’ll want a better failure report for our test job. That way when there’s a failure we won’t have to sift through the console output for info. Instead we should get it all in a formatted report. For that, we’ll turn to JUnit XML (a standard format that CI servers support).

This functionality doesn’t come built into RSpec, but it’s simple enough to add through the use of another gem. There are plenty to choose from with RSpec, but we’ll go with rspec_junit_formatter.

After we install the gem we need to specify some extra command-line arguments when running our tests. A formatter type (e.g., --format RspecJunitFormatter) and an output file for the XML (e.g., --out results.xml). And since this type of output is really only useful when running on our CI server, we’ll want an easy way to turn it on and off.

# filename: .rspec

<% if ENV['ci'] == 'on' %>
--format RspecJunitFormatter
--out tmp/result.xml
<% end %>

Within RSpec comes the ability to specify command line arguments that are used frequently in a file (e.g., .rspec) that lives in the root of the test directory. In it we specify the new commands we want to use and wrap them in a conditional that checks an environment variable that denotes whether or not the tests are being run on a CI server (e.g., if ENV['ci'] == 'on').

Now it’s a small matter of updating our Jenkins job to consume this new JUnit XML output file by adding a post-build action to publish it as a report.

Then we need to tell the Jenkins job where the XML file is. Since it ends up in the root of the test directory, we can just specify the file extension with a wildcard.

Lastly, we need to update the shell commands for the build to set the ci environment variable to on.

Now when we run our test, we’ll get a test report which states which test failed. And when we drill into it, we get the URLs for the jobs in Sauce Labs and Applitools Eyes.

One More Thing: Notifications

In order to maximize your CI effectiveness, you’ll want to send out notifications to alert your team members when there’s a failure.

There are numerous ways to go about this (e.g., e-mail, chat, text, co-located visual cues, etc). And thankfully there are numerous, freely available plugins that can help facilitate whichever method you want. You can find out more about Jenkins’ plugins here.

For instance, if you wanted to use chat notifications and you use a service like HipChat or Slack, you would do a plugin search and find one of the following plugins:

After installing the plugin for your chat service, you will need to provide the necessary information to configure it (e.g., an authorization token, the channel/chat room where you want notifications to go, what kinds of notifications you want sent, etc.) and then add it as a Post-build Action to your job (or jobs).

Now when your CI job runs and fails, a notification will be sent to the chat room you configured.

Outro

If you’ve been following along through this whole series, then you should now have a test that leverages Selenium fundamentals, that performs visual checks (thanks to Applitools Eyes), which is running on whatever browser/operating system combinations you care about (thanks to Sauce Labs), and running on a CI server with notifications being sent to you and your team (thanks to CloudBees).

This is a powerful combination that will help you find unexpected bugs (thanks to the automated visual checks) and act as a means of collaboration for you and your team.

And by using a CI Server you’re able to put your tests to work by using computers for what they’re good at – automation. This frees you up to focus on more important things. But keep in mind that there are numerous ways to configure your CI server. Be sure to tune it to what works best for you and your team. It’s well worth the effort.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Automating Your Test Runs with Continuous Integration — CI Series by Dave Haeffner: Part 3/3 appeared first on Automated Visual Testing | Applitools.