Reviewing results with Selenium IDE

When you run a test using Selenium IDE, you can look at both the script panel and the log panel to get a quick idea as to the outcome of the test run. Typically, you then use the The Test Manager to review the results images, accept or reject mismatches, update the baseline and take other actions to deal with the results.

This article describes the features available in the IDE to help you get quick feedback on the results of the test and to open the Eyes Test Manager to the correct test results, the sections that follow explain:
  • What visual test information you can see in the IDE while a test is running.
  • What test results you can see in the IDE when the test completes.
  • How to automatically open the Eyes Test Manager with the test results
  • How to manually open the Eyes Test Manager from the IDE for a given test result

 

Viewing test progress in the IDE

While a Selenium IDE test is running, you can see the following information that indicates the progress of the test:

  • Each Eyes check command in the script window turns orange when it starts to execute and turns green or red when the result of the matching of that check is known. Green means that the checkpoint matches the corresponding baseline image, or that it is a new checkpoint. Red indicates that the image did not match the baseline image.
  • When a command starts executing, it is displayed in the log pane. When a checkpoint starts to execute, and while the result is not known, it is flagged with an orange Undetermined flag. When the result is known, if the checkpoint matches the baseline, or if it is a new checkpoint, then it is flagged with a green OK. If the match fails, then it is flagged with a Red Failed flag.

Viewing test results in the IDE

When the test competes, In both the script and the log panes you can see the status of each checkpoint as described in the previous section. In addition, you can see a summary, after the last command. If the test passes you will see in the log text similar to:

If one or more checkpoints have failed you will see in the log the text similar to:

In either case, the Log includes a link. If you click on the link, then the Eyes Test Manager is opened in a browser loaded with the test results associated with that link.

Note that if a step is missing, i.e., there is a checkpoint in the baseline, and there is no corresponding checkpoint in the test, then "diffs were found" is displayed and the test ends with an error, but all your checkpoints can still be OK since those checkpoints did pass.

Opening Eyes Test Manager at test completion

You can configure SIDE Eyes so that when the test run completes, the IDE automatically opens Eyes Test Manager, preloaded with the results of that test. For details how to enable or disable this option see The Tests tab

Opening Eyes Test Manager for a given test result

When a test run completes, SIDE Eyes writes a clickable link into the Log panel. Click the link to open the results in the Test Manager.