When you see technical assessment as one of the stages of an interview, what do you do? Does the panic set in? In order to gain confidence in technical tasks, the best way is to practice and tackle them head on.

In this blog, I will walk through:

• My experience as a candidate, having previously had job titles including QA Engineer, SDET, and QA Automation Lead.
• Encountering different types of technical tasks, from the classic whiteboard FizzBuzz programming task to a take-home task building a test framework from scratch.
• Advice as a hiring manager from my experience as QA Lead at a healthcare startup in the UK, where I will touch on not only the technical aspects of the interview but also the behavioral and situational based questions we ask candidates, providing tips on how to prepare as well.

All tips and advice are based on my experience as a hiring manager, hiring for a QA Engineer.

My experience as a candidate

I’ve never been the most technical person. I graduated university with a psychology degree and managed to land a graduate scheme job which taught me the skills on the job, alongside working as a Test Analyst. Therefore, whenever I see a technical part of the job interview process, my anxiety definitely sets in. However, having been through quite a few of these during my career I’ve come up with a few different tactics.

My approach is to refresh my skills either in the programming language or automation test framework and practice! This may mean accepting multiple job interviews in order to use some companies’ technical assignments just as a form of rehearsal. If that’s not possible, Test Automation University (TAU) provides code samples and assignments to refresh your skills.

Of course, every job has its own spin on the technical task but generally follows a similar pattern. Interviews for QA Engineer and Test Automation roles often focus on end-to-end testing with frameworks based on Selenium, Puppeteer, Cypress, or Playwright. While doing these tasks, I always spent too long on them and would focus on using the latest language features or making sure to abstract any duplicate code to helper functions.

Some of the tasks I encountered as a candidate, I definitely thought I had failed as I completed them, especially when they were whiteboard technical tasks. The first one was FizzBuzz and another was sorting and filtering a list. It’s very difficult for me to perform these tasks on a whiteboard using pseudocode without having a keyboard in front of me and without Stack Overflow. Often the person interviewing uses these types of tasks to understand your thought process and how you communicate throughout the activity.

Advice from my candidate experience

These tasks often don’t relate to the daily activities an automation tester or SDET will be performing. In my opinion, the interviewer shouldn’t be assessing the completion of the task or the actual solution. From my experience, my advice for these type of programming tasks: 

• Don’t overthink it: Get a solution down and rewrite it later.
• Ask questions: It can buy some time for thinking.
• Become comfortable with silence: The interviewer won’t be comfortable with the silence either.

The technical assessment

As a hiring manager, I have seen some bad and some bizarre submissions as part of the technical test. Some red flags, I have witnessed while reviewing our technical tasks:

• One of the tasks was missing from the submission
• No assertions in automated checks
• Assertion within test wrapped in try / catch
• “Negative” scenario provided is actually a positive scenario

Getting the basics right is so important. TAU has many courses to help refresh and upskill in preparation for technical jobs.

How I evaluate technical assessments

I will walk through the process of how I evaluate a technical task, which will help how to approach the task effectively:

• Firstly, once I have received the technical task back from the candidate, I open up the README.md file and try to run the project. If the instructions are clear and easy to understand, this gives a great first impression. Making the hiring manager’s job easier by describing how to run the tests is a really good start.
• Secondly, by running the tests (hopefully, they are green), I read the names given to the tests. This defines what is actually being tested, demonstrating understanding of the task itself.
• Thirdly, looking at the new code, have the assertions answered the right question? Especially around the negative scenario, does it handle unwelcome behavior? Too often, negative testing is misunderstood.
• Fourthly, when the test is focused on UI test automation, what element locators have been used? Following the hierarchy of best practices in Cypress documentation is a good reference. Obviously, the best locators are not always available to the candidate, but commenting what should be used will not go unnoticed. This is where comments in the code are allowed and should definitely be used to assist with helping the hiring manager understand your thoughts.
• Fifthly, has the code been structured to facilitate maintenance and stability? For example, moving setup steps to beforeEach proves you know how to make tests independent. Also, refactoring code to be easier to read and understand helps in proving your experience of working with unfriendly code.

These are some key areas which I focus on during the review process, but overall, I’m looking for simplicity, clarity, and following best practices. I always request candidates don’t take too long on a task. Comments with future improvements are enough in this scenario.

The behavioral assessment

Often the interview will be split into multiple sections, one of those could be behavioral or situational style questions. For example, “How do you deal with a situation when a developer says a bug is actually a feature?” The role as an automation tester involves working as part of a team, therefore it’s important to prepare for these questions. As before with the coding exercises, practice can help prepare for this style of interview questions. By rehearsing examples from your experience, the answers are often articulated more fluently.

How I evaluate behavioral assessments

If we take the example of dealing with challenging developers, questioning bugs. Some things I look for:

• Firstly, how closely a tester works with developers and what kind of relationship they have. Working at a startup, it is very important that the QA Engineers and developers work together to solve problems.
• Secondly, persuading and influencing peers. Whether they involve other stakeholders and how much information is gathered before presenting the argument. Again, within a startup environment, we are looking for people who solve their own problems. Involving managers and other stakeholders is still appropriate in certain circumstances, but trying to resolve on your own first shows proactivity and independence.
• Thirdly, attention to detail when it comes to acceptance criteria and what stage this conversation happens within the software development lifecycle (SLDC). Particularly what I am looking for is someone who promotes “3 Amigos” (ideally all 3 but 2 is good enough). These 3 Amigos conversations help eradicate requirements being misunderstood before development starts.

These behavioral or situational questions relate to daily activities for a tester, how someone works within a team, and especially their communication skills. Obviously as a hiring manager, I want to hear about real experiences candidates have had. However, including your opinion on how the team or process could be improved is also valued. Describing the kind of environment the candidate would like to work in helps differentiate between previous and desired experience.

Tips from a hiring manager

Having interviewed many candidates and reviewed lots of technical assessments, these are a few of my tips to think about when interviewing:

1. Don’t be afraid to ask questions. A core attribute of a good tester is asking good questions. Therefore this is encouraged within my interviews, the more questions or clarifications the better.
2. Show your workings. Just like when you are doing a math exam at school. It’s important how you got to the answer, whether that is as comments within code or verbally when explaining your solution.
3. Admit what you don’t know. It’s better to state what you are unsure about, whereas trying to guess, the interviewer can only interpret what you say. An honest person is very well received from my perspective. As I can teach a skill to a prospective candidate, the other attributes are more difficult to coach.

To conclude

As a hiring manager, I am not looking for the finished article. Everyone has had different experiences and opportunities. This should always be taken into consideration. What’s important to demonstrate within the interview process is how you communicate, work as part of a team, and your technical skills. In order to do that, explain your thought process, provide your opinion, and be clear what you still need to learn.

ICYMI: Get connected, be inspired, and sharpen your skills with on-demand expert session recordings from Test Automation University Conference 2023.