How To Get Started with Automated Web Testing — CI Series by Dave Haeffner : Part 1/3

The Problem

If you’re new to automated web testing there’s only a few things you need to know to become effective quickly. But unless someone tells you what they are you could easily head down the wrong path and end up wasting time on things which will yield very few results.

A Solution

By learning the fundamentals of Selenium (a popular free and open-source browser automation tool), you can be up and running with cross-browser automated web tests rather quickly. And when combined with a third-party automated visual testing solution like Applitools Eyes, you can have an impressively high amount of test coverage with surprisingly little effort.

An Example

Selenium is built to mimic human action and it works with two pieces of information to do it: the element on the page you want to interact with and the action you want to take on that element.

If we take a common example like login for a website, here are the actions Selenium would take to step through it like a user would:

• visit the page
• find the username input field and type text into it
• find the password input field and type text into it
• find the submit button and click it

Let’s take the login example found on the-internet and write a Selenium test for it. Let’s use a scripting language (e.g., Ruby), since it is very approachable and reads a lot like English. And we’ll use RSpec, which is another open source library. It will enable us to organize our tests easily and perform repeatable actions before and after each test.

Here is what the initial Selenium test looks like once written:

# filename: login_spec.rb

require 'selenium-webdriver'

describe 'Login' do

  before(:each) do
    @driver = Selenium::WebDriver.for :firefox
  end

  after(:each) do
    @driver.quit
  end

  it 'succeeded' do
    @driver.get 'http://the-internet.herokuapp.com/login'
    @driver.find_element(id: 'username').send_keys 'tomssmith'
    @driver.find_element(id: 'password').send_keys 'SuperSecretPassword!'
    @driver.find_element(css: 'button').click
  end
  
  end

NOTE: RSpec has some syntax that is worth pointing out. A test case starts with the word describe, tests start with the word it followed by the test name (e.g., it 'succeeded' do`), and you can specify actions to occur before and after each test with before(:each) and after(:each). The words do and end are Ruby code which signify the beginning and end of a chunk of code (a.k.a. a block). And test files in RSpec are known as “specs”, which need to end with _spec.rb in the filename (e.g., login_spec.rb).

At the top of the file we pull in the Selenium Ruby bindings (e.g., require 'selenium-webdriver') and declare our test case (e.g., describe 'Login' do). Before each test we will create an instance of Firefox and store it in an instance variable to be used during and after our test (e.g., @driver = Selenium::WebDriver.for :firefox). After each test we will close the browser (e.g., @driver.quit).

Next, we create our login test and fill it with our Selenium commands (which all work using the @driver variable created in before(:each)).

The first command of the test visits the login page with .get followed by the URL as a string (e.g., 'http://the-internet.herokuapp.com'). We then interact with the login form on the page by finding the username input field with .find_element and a locator for the element (e.g., (id: 'username')) along with the action we want to take (e.g., .send_keys 'tomsmith'). And we repeat the same approach again for the password field (e.g., @driver.find_element(id: 'password').send_keys 'SuperSecretPassword!'). For the last command we issue one more .find_element to find the submit button (e.g., (css: 'button')) and click it with .click.

If we save this file and run it (e.g., rspec login_spec.rb from the command-line), it will open the browser and complete the login on the page. But it wouldn’t tell us if the application behaved as expected. To handle this we need to add in automated visual checks with Applitools Eyes.

The Eyes Have It

NOTE: To use Applitools Eyes you need to grab a free account from here (no credit card required).

In order to incorporate Applitools Eyes into our test, we first need to make some modifications to our test setup and teardown.

# filename: login_spec.rb

require 'selenium-webdriver'
require 'eyes_selenium'

describe 'Login' do

  before(:each) do
    @browser = Selenium::WebDriver.for :firefox
    @eyes = Applitools::Eyes.new
    @eyes.api_key = ENV['APPLITOOLS_API_KEY']
    @driver = @eyes.open(app_name: 'the-internet', test_name: 'login', driver: @browser)
  end
# ...
  after(:each) do
    @eyes.abort_if_not_closed
    @driver.quit
  end
# ...

Once we pull in the Applitools Eyes Ruby SDK (e.g., require 'eyes_selenium') we update our test setup (e.g., before(:each)) by creating an instance of Applitools Eyes (storing it in another instance variable for later use), specifying our API key, and starting an Applitools Eyes session (which requires the name of the app, the name of the test, and the Selenium instance). After each test we make sure to abort the Eyes session if it was unable to close properly (e.g., @eyes.abort_if_not_closed) – more on that soon. And we do this before destroying the Selenium instance (since the Eyes instance relies on Selenium).

NOTE: In this example the API key is being passed through an environment variable. You can just as easily hard-code your API key value here.

Now we’re ready to add some visual checks to our test so we can verify that our application works as intended (and verify that the page renders correctly).

# ...
  it 'succeeded' do
    @driver.get 'http://the-internet.herokuapp.com/login'
    @eyes.check_window('Login Page')
    @driver.find_element(id: 'username').send_keys 'tomsmith'
    @driver.find_element(id: 'password').send_keys 'SuperSecretPassword!'
    @driver.find_element(css: 'button').click
    @eyes.check_window('Logged In')
    @eyes.close
  end

end

With the @eyes.check_window command we’re performing visual checks. We can specify a value with it or not (e.g., ('Login Page'), ('Logged In')). The value is optional, but it will appear in the test results in Applitools Eyes. It could add a helpful narrative for you (or other people on your team) to follow along with what the test was doing when the results are reviewed after the fact.

@eyes.close ends the Applitools Eyes session and triggers the final comparison of the visual checks.

Expected Behavior

When you save this file and run it (e.g., rspec login_spec.rb from the command-line) here is what will happen:

• Selenium opens the browser
• An Applitools Eyes session is created and connected with the Selenium instance
• The test runs the Selenium commands and captures visual checks along the way
• The Applitools Eyes session closes and performs validation on the visual checks
• Selenium closes the browser
• Failures (if any) are displayed in the console output

If Applitools Eyes finds a visual anomaly it will fail the test and provide a URL to the test results in the console output. When viewing the results in Applitools Eyes you can choose to accept or reject the results, which will impact the way Eyes validates the test going forward.

NOTE: A new test in Applitoosl Eyes will fail and prompt you to view the results. It’s recommended that you review the results, but it’s not mandatory. The results will automatically be saved as a baseline image for subsequent runs.

A Small Bit of Cleanup

In order to make this example useful for more than one test, you need to make a small tweak to your test setup.

# filename: login_spec.rb
# ...
  before(:each) do |example|
    @browser = Selenium::WebDriver.for :firefox
    @eyes = Applitools::Eyes.new
    @eyes.api_key = ENV['APPLITOOLS_API_KEY']
    @driver = @eyes.open(app_name: 'the-internet', test_name: example.full_description, driver: @browser)
  end
# ...

Within RSpec you can access each test in the before(:each) by adding a block variable (e.g.,before(:each) do |example|). You can then put it to use @eyes.open when specifying the test name (e.g., test_name: example.full_description).

Now when you add another test to this file the name will be dynamically passed to Applitools Eyes.

Outro

By combining your Selenium tests with an automated visual testing solution you are armed with a powerful combination that gives you a staggering level of coverage (e.g., hundreds of assertions) for only a few lines of code. And as a bonus you’ve automated something that many people think can only be done manually (and as a result tends to get relegated to the end of a software development cycle) – moving you and your team one massive step closer to Continuous Delivery. Bravo!

In PART 2 of this series: “How To Run Your Automated Web Tests From Any Browser”, I’ll show you how to run your tests against whatever browser and operating system combination you need.

You can read the third and final post in this series: “Automating Your Test Runs with Continuous Integration”.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.