In one of my recent articles, How Visual UI Testing can speed up DevOps flow<\/a> I showed how you can configure a CD\/CI service to run your Visual UI tests. The end result would be the same, whether you are running your tests locally, or via such services. Once the tests run, you can always check the results over the Applitools Test Manager Dashboard.<\/p>\n\n\n\n In this article, I will show you how you can run your Visual UI tests, whether you\u2019ve written them for your mobile or web app, on real physical mobile devices in the cloud. For this, I will be employing Applitools<\/a>, Appium<\/a>, and AW<\/a>S Device Farm<\/a>.<\/p>\n\n\n\n AWS Device Farm is a mobile app testing platform that helps developers automatically test their apps on hundreds of real devices in minutes.<\/p>\n\n\n\n When it comes to testing your app over mobile devices, the choices are numerous. Amazon helps to build a \u201cDevice Farm\u201d on behalf of the developers and testers, hence the name.<\/p>\n\n\n\n Here are some of the major advantages and features for using this service:<\/p>\n\n\n\n AWS Device Farm supports a number of test runners. This includes Appium Java JUnit, Appium Python, Appium Ruby, and Appium Java TestNG. Back in January 2019, Amazon announced<\/a> support for the Appium Node.js test runner. This means you can build your tests with Selenium Webdriver, for instance, and have it run on top of AWS Device Farm.<\/p>\n\n\n\n Now that you have an idea about AWS Device Farm, let\u2019s move on, and discover the Appium automation testing framework.<\/p>\n\n\n\n Selenium WebDriver<\/a> is a browser automation framework that allows a developer to write commands, and send them to the browser. It offers a set of clients with a variety of programming languages (Java, JavaScript, Ruby, Python, PHP and others). <\/p>\n\n\n\n Figure 1<\/strong> below shows the Selenium WebDriver architecture:<\/p>\n\n\n\n Figure 1: Selenium WebDriver Architecture<\/strong><\/p>\n\n\n\n Selenium WebDriver architecture consists of:<\/p>\n\n\n\n Selenium 4<\/a> is obseleting the JSONWP in favor of the new W3C WebDriver standard.<\/p>\n\n\n\n Here\u2019s a quick tutorial on using and learning Selenium WebDriver<\/a>.<\/p>\n\n\n\n With that brief overview of Selenium WebDriver, let\u2019s move on and explore Appium.<\/p>\n\n\n\n Appium <\/a>is an open-source tool to automate Mobile app testing. It\u2019s a cross-platform that supports both OS (Android and iOS) test scripts. It is tested on simulators (iOS), emulators (Android) and real devices (iOS, Android).<\/p>\n\n\n\n It\u2019s an HTTP Server written in Node.js that creates and handles WebDriver sessions. When you install Appium, you are actually installing the Appium Server. It follows the same approach as the Selenium WebDriver, which receives HTTP requests from the Client Libraries in JSON format with the help of JSONWP. It then handles those HTTP Requests in different ways. That\u2019s why you can make use of Selenium WebDriver language bindings, client libraries and infrastructure to connect to the Appium Server. <\/p>\n\n\n\n Instead of connecting a Selenium WebDriver to a specific browser WebDriver, you will be connecting it to the Appium Server. Appium uses an extension of the JSONWP called the Mobile JSON Wire Protocol<\/a> (MJSONWP) to support the automation of testing for native and hybrid mobile apps. <\/p>\n\n\n\n It supports the same Selenium WebDriver clients <\/a>with a variety of multiple programming languages such as Java, JavaScript, Ruby, Python, PHP and others. <\/p>\n\n\n\n Being a Node.js HTTP Server, it works in a client-server architecture. Figure 2<\/strong> below depicts the Appium Client-Server Architecture model:<\/p>\n\n\n\n Figure 2: Appium Server Architecture<\/strong><\/p>\n\n\n\n Appium architecture consists of:<\/p>\n\n\n\n The results of the test session are then communicated back to the Appium Server, and back to the Client in the form of logs, using the Mobile JSONWP.<\/p>\n\n\n\n Now that you are well equipped with knowledge for Selenium WebDriver and Appium, let\u2019s go to the demo section of this article.<\/p>\n\n\n\n In this section, we will write a Visual UI test script to test a Web page. We will run the tests over an Android device both locally and on AWS Device Farm. <\/p>\n\n\n\n I will be using both Selenium WebDeriver and Appium to write the test script.<\/p>\n\n\n\n Before you can start writing and running the test script, you have to make sure you have the following components installed and ready to be used on your computer:<\/p>\n\n\n\n Assuming you are working on a MacOS computer, you can verify the above installations by running the following bash commands:<\/p>\n\n\n\n For this demo we need to install Appium Server, Android Studio \/ SDK and finally make sure to have a few environment variables properly set.<\/p>\n\n\n\n Let\u2019s start by installing Appium Server. Run the following command to install Appium Server locally on your computer.<\/p>\n\n\n\n The command installs the Appium NPM package globally on your computer. To verify the installation, run the command:<\/p>\n\n\n\n Now let\u2019s install Android Studio \/ SDK so that you can run the test script on an emulator or real device. You could install the Android SDK only but then you have to do additional advanced steps to properly configure the Android environment on your computer. I highly recommend installing the Android Studio as it makes your life easier.<\/p>\n\n\n\n Download the Android Studio executable<\/a>. Follow the steps below to install locally on your computer:<\/p>\n\n\n\n Notice the location where the Android SDK was installed. It\u2019s \/Users\/{User Account}\/Library\/Android\/sdk<\/strong>.<\/p>\n\n\n\n Wait until the download and installation is complete. That\u2019s all!<\/p>\n\n\n\n Because I want to run the test script locally over an Android emulator, let\u2019s add one.<\/p>\n\n\n\n Open the Android Studio app:<\/p>\n\n\n\n Click the Configure<\/strong> icon:<\/p>\n\n\n\n Select the AVD Manager<\/strong> menu item.<\/p>\n\n\n\n Click the + Create Virtual Device<\/strong> button.<\/p>\n\n\n\n Locate and click the Pixel XL<\/strong> device then hit Next.<\/strong><\/p>\n\n\n\n Locate the Q<\/strong> release and click the Download<\/strong> link.\u00a0<\/p>\n\n\n\n Read and accept the Terms and Conditions then hit Next.<\/strong><\/p>\n\n\n\n The Android 10, also known as Q release, starts downloading.<\/p>\n\n\n\n Once the installation is complete, click the Next <\/strong>button to continue setting up an Android device emulator.<\/p>\n\n\n\n The installation is complete. Grab the AVD Name<\/strong> as you will use it later on in the test script, and hit Finish.<\/strong><\/p>\n\n\n\n Finally, we need to make sure the following environment variables are set on your computer. Open the ~\/.bash_profile<\/strong> file, and add the following environment variables:<\/p>\n\n\n\n Finally, add the above environment variables to the $PATH as follows:<\/p>\n\n\n\n One last major component that you need to download, and have on your computer, is the ChromeDriver. Navigate to the Appium ChromeDriver<\/a> website, and download the latest workable ChromeDriver release for Appium. Once downloaded, make sure to move the file to the location: \/usr\/local\/bin\/chromedriver<\/strong><\/p>\n\n\n\n That\u2019s it for the installations! Let\u2019s move on and explore the Visual UI test script in depth.<\/p>\n\n\n\n You can find the source code demo of this article on this GitHub repo<\/a>.<\/p>\n\n\n\n Let\u2019s explore the main test script in this repo.<\/p>\n\n\n\n The test script starts by importing the selenium-webdriver<\/a> NPM package.<\/p>\n\n\n\n It imports a bunch of objects from the @applitools\/eyes-selenium<\/a> NPM package.<\/p>\n\n\n\n It constructs a BatchInfo<\/a> object used by Applitools API. <\/p>\n\n\n\n It then creates the Eyes<\/strong> object that we will use to interact with the Applitools API.<\/p>\n\n\n\n It\u2019s so important to set the Applitools API key at this stage. Otherwise, you won\u2019t be able to run this test. The code above also directs the Applitools API logs to a File located at the root of the project under the name of eyes.log<\/strong>.<\/p>\n\n\n\n Next, we define the device capabilities <\/strong>that we are going to send to Appium.<\/p>\n\n\n\n We are using an Android emulator to run our test script over a Chrome browser with the help of the UIAutomator 2<\/strong> library.<\/p>\n\n\n\n We need to set the avd<\/strong> capability only when running this test script locally. For this property, grab the AVD ID of the Android Device Emulator we set above.<\/p>\n\n\n\n Now, we create and build a new WebDriver<\/strong> object by specifying the Appium Server local URL and the device capabilities as:<\/p>\n\n\n\n Appium is configured to listen on Port 4723<\/strong> under the path of \/wd\/hub<\/strong>.<\/p>\n\n\n\n The rest of the script is usual Applitools business. In brief, the script:<\/p>\n\n\n\n Notice that the script asserts two Eyes SDK Snapshots. The first captures the home page of the website, while the second captures the Location section.<\/p>\n\n\n\n Finally, some important cleanup is happening to close the WebDriver and Eyes SDK sessions.<\/p>\n\n\n\n Open the package.json<\/strong> file, and locate the two scripts there:<\/p>\n\n\n\n The first runs and starts the Appium Server, and the second to run the test script.<\/p>\n\n\n\n Let\u2019s first run the Appium server by issuing this command:<\/p>\n\n\n\n Then, once Appium is running, let\u2019s run the test script by issuing this command:<\/p>\n\n\n\n Login to the Applitools Test Manager located at: https:\/\/applitools.com\/users\/login<\/a><\/p>\n\n\n\n You will see the following test results:<\/p>\n\n\n\n The two snapshots have been recorded!<\/p>\n\n\n\n Now that the test runs locally, let\u2019s run it on AWS Device Farm. Start by creating a new account on Amazon Web Service website.<\/p>\n\n\n\nAWS Device Farm for mobile visual testing<\/h2>\n\n\n\n
\n
\n
\n
\n
\n
Selenium Webdriver for browser app automation<\/h2>\n\n\n\n
![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_1-1024x449.png\")
<\/figure>\n\n\n\n\n
\n
\n
\n
Appium for mobile app automation<\/h2>\n\n\n\n
![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_2-1024x558.png\")
<\/figure>\n\n\n\n\n
\n
\n
\n
Demo<\/h2>\n\n\n\n
Prerequisites<\/h3>\n\n\n\n
\n
echo $JAVA_HOME \/\/ this should print the Java SDK path
node -v \/\/ this should print the version of Node.js installed
npm -v \/\/ this should print the version of the Node Package Manager installed<\/pre>\n\n\n\nComponent Installations<\/h3>\n\n\n\n
npm install -g appium<\/pre>\n\n\n\n
appium -v \/\/ this should print the Appium version<\/pre>\n\n\n\n
![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_3-1024x799.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_4-1024x799.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_5-1024x799.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_6-1024x799.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_7-1024x799.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_8-1024x782.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_9.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_10-1024x782.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_11-1024x722.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_12-1024x794.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_13-1024x794.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_14-1024x722.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_15-1024x722.png\")
<\/figure>\n\n\n\nAPPLITOOLS_API_KEY={Get the Applitools API Key from Applitools Test Manager}<\/strong>
export APPLITOOLS_API_KEY
ANDROID_HOME=\/Users\/{Use your account name here}<\/strong>\/Library\/Android\/sdk
export ANDROID_HOME
ANDROID_HOME_TOOLS=$ANDROID_HOME\/tools
export ANDROID_HOME_TOOLS
ANDROID_HOME_TOOLS_BIN=$ANDROID_HOME_TOOLS\/bin
export ANDROID_HOME_TOOLS_BIN
ANDROID_HOME_PLATFORM=$ANDROID_HOME\/platform-tools
export ANDROID_HOME_PLATFORM
APPIUM_ENV=\"Local\"
export APPIUM_ENV<\/pre>\n\n\n\nexport $PATH=$PATH:$ANDROID_HOME:$ANDROID_HOME_TOOLS:$ANDROID_HOME_TOOLS_BIN:$ANDROID_HOME_PLATFORM<\/pre>\n\n\n\n
Run the Visual UI Test Script locally<\/h3>\n\n\n\n
\"use strict\";\n\n;(async () => {\n\n const webdriver = require(\"selenium-webdriver\");\n const LOCAL_APPIUM = \"https:\/\/web.archive.org\/web\/20221206000829\/http:\/\/127.0.0.1:4723\/wd\/hub\";\n\n \/\/ Initialize the eyes SDK and set your private API key.\n const { Eyes, Target, FileLogHandler, BatchInfo, StitchMode } = require(\"@applitools\/eyes-selenium\");\n\n const batchInfo = new BatchInfo(\"AWS Device Farm\");\n batchInfo.id = process.env.BATCH_ID\n batchInfo.setSequenceName('AWS Device Farm Batches');\n \n \/\/ Initialize the eyes SDK\n let eyes = new Eyes();\n eyes.setApiKey(process.env.APPLITOOLS_API_KEY);\n eyes.setLogHandler(new FileLogHandler(true));\n eyes.setForceFullPageScreenshot(true)\n eyes.setStitchMode(StitchMode.CSS)\n eyes.setHideScrollbars(true)\n eyes.setBatch(batchInfo);\n\n const capabilities = {\n platformName: \"Android\",\n deviceName: \"Android Emulator\",\n automationName: \"UiAutomator2\",\n browserName: 'Chrome',\n waitforTimeout: 30000,\n commandTimeout: 30000,\n };\n\n if (process.env.APPIUM_ENV === \"Local\") {\n capabilities[\"avd\"] = 'Pixel_XL_API_29';\n }\n \n \/\/ Open browser.\n let driver = new webdriver\n .Builder()\n .usingServer(LOCAL_APPIUM)\n .withCapabilities(capabilities)\n .build();\n\n try {\n \/\/ Start the test\n await eyes.open(driver, 'Vuejs.org Conferences', 'Appium on Android');\n\n await driver.get('https:\/\/us.vuejs.org\/');\n\n \/\/ Visual checkpoint #1.\n await eyes.check('Home Page', Target.window());\n\n \/\/ display title of the page\n await driver.getTitle().then(function (title) {\n console.log(\"Title: \", title);\n });\n\n \/\/ locate and click the burger button\n await driver.wait(webdriver.until.elementLocated(webdriver.By.tagName('button.navbar__burger')), 2000).click();\n \n \/\/ locate and click the hyperlink with href='\/#location' inside the second nav element\n await driver.wait(webdriver.until.elementLocated(webdriver.By.xpath(\"\/\/web.archive.org\/web\/20221206000829\/https:\/\/nav[2]\/ul\/li[3]\/a[contains(text(), 'Location')]\")), 2000).click();\n\n const h2 = await driver.wait(webdriver.until.elementLocated(webdriver.By.xpath(\"(\/\/h2[@class='section-title'])[4]\")), 2000);\n console.log(\"H2 Text: \", await h2.getText());\n\n \/\/ Visual checkpoint #2.\n await eyes.check('Home Loans', Target.window());\n\n \/\/ Close Eyes\n await eyes.close();\n } catch (error) {\n console.log(error);\n } finally {\n \/\/ Close the browser.\n await driver.quit();\n\n \/\/ If the test was aborted before eyes.close was called, ends the test as aborted.\n await eyes.abort();\n }\n})();<\/code><\/pre>\n\n\n\nconst batchInfo = new BatchInfo(\"AWS Device Farm\");\nbatchInfo.id = process.env.BATCH_ID\nbatchInfo.setSequenceName('AWS Device Farm Batches');<\/pre>\n\n\n\n\/\/ Initialize the eyes SDK
let eyes = new Eyes();
eyes.setApiKey(process.env.APPLITOOLS_API_KEY);
eyes.setLogHandler(new FileLogHandler(true));
eyes.setForceFullPageScreenshot(true)
eyes.setStitchMode(StitchMode.CSS)
eyes.setHideScrollbars(true)
eyes.setBatch(batchInfo);<\/pre>\n\n\n\nconst capabilities = {
platformName: \"Android\",
deviceName: \"Android Emulator\",
automationName: \"UiAutomator2\",
browserName: 'Chrome',
waitforTimeout: 30000,
commandTimeout: 30000,
};
if (process.env.APPIUM_ENV === \"Local\") {
capabilities[\"avd\"] = 'Pixel_XL_API_29';
}<\/pre>\n\n\n\nconst LOCAL_APPIUM = \"http:\/\/127.0.0.1:4723\/wd\/hub\";
let driver = new webdriver
.Builder()
.usingServer(LOCAL_APPIUM)
.withCapabilities(capabilities)
.build();<\/pre>\n\n\n\n\n
\"appium\": \"appium --chromedriver-executable \/usr\/local\/bin\/chromedriver --log .\/appium.log\",
\"test\": \"node appium.js\"<\/pre>\n\n\n\nnpm run-script appium<\/pre>\n\n\n\n
npm run-script test<\/pre>\n\n\n\n
Verify Test Results on Applitools Test Manager<\/h3>\n\n\n\n
![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_16-1024x690.png\")
<\/figure>\n\n\n\nRun the Visual UI Test Script on AWS Device Farm<\/h3>\n\n\n\n
![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_17-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_18-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_19-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_20-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_21-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_22-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_23-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_24-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_25-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_26-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_27-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_28-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_29-1024x690.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/applitools.com\/wp-content\/uploads\/2023\/12\/amazon_visual_testing_30-1024x690.png\")
<\/figure>\n\n\n\n