Testing iFrames With Visual AI

Actually, tracking and analytics make up many reasons why you might install iFrames. While they can look seamless on your app, they can call external functions – including third-party tracking tools. But, you can embed almost any functionality using iFrames.

Like, say, a Google Map.

A Quick Google Map Example

The code to embed a Google Map is an iFrame:

Yup, this code will embed a Google Map of Alice Springs, Northern Territory, Australia. Someplace I’ve always wanted to visit. I haven’t made it, yet.

You can use the tracking pieces to know where your customers go on your web app. In fact, some of these tracking iFrames are zero pixel iFrames, meaning they are invisible on your page.  The existence of zero pixel iFrames cannot be detected in your end-user and integration testing. They will exist in your DOM, but end users won’t know about their existence.

What matters, then, are the pages that include iFrames for extended functionality.  They are embedded in your web app – sometimes nested. How do you test an iFrame?

Possible iFrame Errors

iFrames can be built to the size of the user’s screen, or any subset of your app page. As I mentioned, some are one or zero pixels high. The challenge comes around building interactive iFrames.

When you test iFrames designed for users interaction, the iFrame can behave as if is part of your app. Buttons and functionality inside the iFrame will need to be tested.  These iFrames have their own contents, buttons, and scrolling regions.  They have their own behaviors that need to be tested.

You will want to test the data and functionality in your iFrames. You will want to ensure that buttons behave as expected, scrolling regions scroll, and expected information gets presented to the user. These behaviors get complicated by how you define your iframe and ensure its boundaries. For some designs, the behavior works fine on a mobile device but gets lost on a desktop or laptop scree. In other designs, the mobile device behavior might not be acceptable.

In this example, Raja points out design considerations that your developers may not have considered when designing your app. The big problems occur with testing iframes sized smaller than their contents – which results in scroll bars on the iFrame to access all the content. Or not – some iFrame implementations are fixed with no scroll bars. How do you test the iFrame?

Testing iFrames with Selenium

Raja imagines a test of a table in a nested iFrame. As the page is resized, the top-level iFrame resizes, and the content in the nested iFrame gets hidden. Depending on the design approach, this hidden data may not become visible unless the user scrolls the data manually. And, in some cases, there may not be scroll bars – meaning a user must manually resize the page to see the data.

Raja walks into the legacy approach to testing iFrames – using Selenium to apply a test behavior and then measure the response of the page. There are several complications he exposes.

Nested iFrame Navigation

The first is the challenge of navigating iFrames – especially if you have nested them. Each time you navigate in an iFrame, you must change the context of the iFrame you’re inspecting. You have to remember your context. And, you have to extract your context to go back to the page outside the iFrame.

In the code that follows, Raja shows the redudnant code you need to add to navigate the different nested iFrames. First, you go to the main page context. Next, you go to the top iFrame. Finally, you select one of the four embedded iFrames. Now, you can begin your tests.

Here is a highlight of the code used to change context:

This is a lot of redundant code needed to navigate around to each iFrame.

Hidden Content

Another complication involves the hidden parts of frames. When pages resize and frame content gets hidden, the user may not know the content exists, or its values.. Hidden content can create problems in testing, because the test may not uncover the error.

And, thus, the issue of scroll bars. If a scroll bar does not exist, can the user see data that does not show up on the page? How does a tester use an automated tool to expose this behavior?

 

Selenium Testing iFrame Application Versions

Selenium can accomplish iFrame resizing and scrolling, but the efforts to do these tasks depend very much on the test engineer. Also, there are dependencies on the version of the application itself. But, validation and assertion technologies limit traditional functional test approaches.

For example, a tester may get a version of the app with scroll bars in a nested iFrame. The engineer might choose not to exercise the scroll bars, and leave that task to the user later. Instead of testing the existence of scroll bars, the tester might simply use text locators to validate that the content exists in the DOM.

If a later version of the app leaves out the scroll bars, then the locators still find the appropriate content, but the user has a much more difficult time accessing the content.  Depending on how you structure your Selenium tests, you might miss this change and pass the test. In fact, all functional testing that uses DOM checking will generally miss the user problem this type of issue poses.

Also, things like nested iFrames cause context problems for both applying an action and asserting an outcome.  Each time a sub iFrame requires an action, Selenium must first switch context to the main iFrame, then the sub iFrame in question.

Raja describes these types of issues with iFrames as generally problematic for legacy functional tests.

Testing iFrames with Visual AI

Because Visual AI uses image comparison to determine the rendered output, the existence of scroll bars does not require any kind of action or assertion.  Instead, if the scroll bars are otherwise invisible, selecting the iFrame should highlight the scroll bars.

In fact, much of iFrame testing with visual AI simply requires taking an action, and then a snapshot of the page. Scroll bars, iFrame contents, and iFrame formatting all can be checked by taking an action along with a snapshot.

Here is a picture of the code from the course.

Complicated assertions become simple implementations of:

eyes.checkWindow(“iFrame”);

Subsequent tests of iFrames can compare visual behavior of the page with its predecessors. Visual AI captures missing scroll bars and other changes in iFrame contents, format, and behavior. You can approve intended behavior changes with a “Thumbs-up”, andy can call out unintended behavior changes (like missing scroll bars and hidden data) with a “Thumbs-down” and a snapshot of both the behavior and the related code.

Because visual assertions replace calls to locators, test engineers have an easier time writing and maintaining test code.

Conclusion

This chapter makes the point that, once again, tasks that can make the life of a test engineer exceedingly difficult with legacy functional tests turn into simple problems for Visual AI.  In this case, you don’t have to instrument every iFrame behavior. Simply use Visual AI and let the visual differences guide your testing.

In all honesty, your mileage may vary with this chapter. I know that some applications make heavy use of iFrames for user interaction, while others only depend on iFrames for user tracking. So, depending on the technology you deploy in your applications, you may or may not benefit from this particular value of Visual AI. But, the point Raja makes is that this is yet another task made easier with Visual AI.

For More Information

• Sign up for Test Automation University
• Read all about Functional Test Myopia
• Modernize Your Functional Testing– Chapter 1 of Raja’s course.
• Advanced Tools for Testing Tables – Chapter 2 of Raja’s course.
• Data-Driven Testing with Visual AI – Chapter 3 of Raja’s course.
• How Do you Test Dynamic Content? – Chapter 4 of Raja’s course.
• Visit the Automated Visual Testing Course on Test Automation University
• Blog: How Do I Validate Visually?
• Sign up for a free Applitools account
• Request an Applitools Demo
• Visit the Applitools Tutorials