Ram Nathaniel joined Applitools in 2017, with a mission to make visual AI and visual testing accessible to all frontend developers and functional testers across all web platforms and devices. To do this, he set about building a team of innovative algorithm developers. They apply computer vision and deep learning techniques to complex and challenging algorithmic problems. Armed with over 20 years experience, Ram is growing his Algorithms team at Applitools, and is dedicated to hiring the most creative and passionate people in the industry.
Ram sat down with us to explain his work and his team.
Hi Ram, thanks for joining us! You’re currently expanding your Computer Vision Algorithms team – could you tell us about the impact algorithm developers have at Applitools?
My team develops the core computer-vision technology of the company.
We developed a computer system to emulate the human eye and brain, look at millions of images of application screens, and use AI-powered computer vision technology to compare them. Our algorithms only report UI differences that are significant and perceptible to the human eye. This unique problem challenges our team.
We want to offer a simple solution to a time-consuming problem that application developers and software testing professionals must overcome every day: look at user interfaces and make sure it is visually correct, and remains visually perfect across a wide matrix of devices, screen sizes, browsers, and operating systems, dynamic content, localized languages, font sizes, and more. Most engineers find this to be laborious and time-consuming work. Even experienced engineers make mistakes in visual validation. My team believes a computer should do this work. We develop sophisticated computer vision algorithms that automate visual validation and allow this task to be performed automatically much faster, and in a much more accurate manner than a human can do.
Think about the number of screens you read your daily news update on. For each device, the same news page is presented in a slightly different display – smaller screens, different pixels, different OS, etc. Our human eye processes these changes without us paying too much notice – that is, until something goes wrong (check out #guigonewrong for some examples!). Our brain figures this out pretty quickly because we encounter challenging images all the time. A computer, on the other hand, needs to be taught what’s visually right or wrong. My team teaches the computer to observe user interfaces, thoroughly understand it, and make sure it is correct and of good quality. Thus eradicating this tiresome job for frontend developers and app testers.
I see automated visual regression testing becoming a must-have tool for anyone writing code. It makes life and work for UI developers more efficient and productive, while also removing the risk of human error. As we progress with our work, I truly believe we are approaching a time when no user interface will be created, without the assistance of visual test automation to visually validate its correctness.
Could you explain some of the challenges that you encounter and how you go about overcoming them?
Applitools’ unique technology enables validating that the structure is still the same, despite the content having changed. This means that the algorithms need to thoroughly understand what they process about each image, in order to separate the dynamic data from the structure.
We must build a computer vision system, introduce these images to the system, train it to recognize each tiny nuance, and validate correctness. This means creating highly accurate algorithms. Our customers trust us with the quality of our product. We strive to deliver more than just high detection rates. We aim for perfection within our tool and the algorithms that support this. It’s not an easy task, but we’re getting closer to achieving it every day.
We must contend with an added layer of complexity – dynamic technology for designing web applications. While we constantly push the envelope of technology to advance our tool, the world of web and mobile app design keeps pushing the envelope of creativity by introducing new and more challenging user interfaces. Software development teams and customers appreciate the innovations, but those innovations present a huge challenge for our team. We must keep up with these changes and help our product navigate and evolve around each new piece of the puzzle.
Because our team drives core product behavior, any behavioral change impacts our customer base, which is growing every day. An algorithmic change that my team makes affects hundreds of customers that run millions of comparisons each week. This central role requires us to focus on perfection and quality-control when we set our deliverables.
We work hard to overcome these challenges. Our role leads to a perfect an artificial sense of sight that fully understands what we present to it. It takes constant work, discovery and reiterations to the process, but my team is up for the challenge!
What are the key traits that your team demonstrate to cope with these challenges?
I have participated in quite a few computer vision projects over the years. Most improved on a well-defined task. Here, we are inventing completely new technology. In my experience, inventive teams require highly skilled, confident, diverse and collaborative members! We seek a very particular skill set, centered around creativity, coupled with a proven track record of persistence and drive.
The people we hire are passionate about what they are doing and are innovators by character. At Applitools, we produce complex vision algorithms that propel computer vision systems in new directions. This means that each person takes ownership of what they do and recognize how it contributes to our overall mission.
My team takes pride in their work and they are fearless when it comes to experimenting with new technologies. In turn, this persistence and creativity motivates others to innovate and collaborate within their roles. This type of creativity revolutionizes an industry and paves the way for Applitools to create the world’s first automated visual UI testing platform.
We solve a dynamic problem that evolves constantly. This is what makes it exciting! We are at the forefront of our field, and when people join us they need to demonstrate this passion and drive. I don’t expect them to know everything on day one. We take the time to teach new joiners the new techniques we have been developing. After this, it is down to their hunger to learn and their ambition to have an impact within the industry. In order to be a developer at the cutting edge of new technology, it takes a bit of a “hutzpa” – to dare and aspire, in order to change the world!
Great! As your team continues to grow, what aspirations do you have for computer vision algorithms at Applitools?
I believe ambitious aspirations are what motivate a team to achieve more with their work. A while ago, we all sat down together and completed an exercise called ‘our future selves’ in which we envisioned where we would be a year from now.
When I look to the future of UI development, I see app developers designing with Applitools products. The Applitools product family provides developers with accurate information, telling them if their graphical language works, if their user experience flows, if they have a valid implementation, across all target screens and devices, and if visually impaired people can still use their software… There is also a lot of scope for us to expand the coverage of the tool so that it can be applied within virtual and augmented reality. One day, I can see Applitools providing a vital tool for all 3D games too.
My team of algorithm developers is at the heart of the visual automation testing revolution. I aspire that their work will continue to blur the line between what is considered work for a human, and work for a computer. Our AI solutions will shape the future of software development and improve the way people interact with software every day. We will continue to push the industry in new directions and fuel the rapid growth of Applitools.
At the inception of Applitools, the founders heard time and again from engineers and investors that the company would find it impossible to build this tool. I am proud to say that my team’s work proves those skeptics wrong. Our vision, working on the cutting edge of this new technology, enables this impossible idea to become a reality.
