Because life's too short to spend it reading terrible interview feedback
This is my rough guide to how to prepare to do good tech interviews and provide useful feedback. It’s not a one-size-fits-all guide; it’s the best practices I’ve adopted after giving more than 400 technical interviews to candidates with all ranges of experience and backgrounds who are applying for roles with a strong software engineering component to them.
It assumes you're an interviewer using the typical tech interview format of a 45-60 minute discussion with a candidate, generally focusing on a specific area of knowledge.
I'm not saying that’s the right way to conduct candidate interviews, or commenting on the various merits of this approach vs. take-home coding interviews, or pair-programming with an interviewer, or any other approach. But if this is the approach your organisation has taken to interviews, this is how you can conduct them well.
Before the interview
I assume:
- You have time to prepare for the interview
- You are familiar with the role and level the candidate is interviewing for, and the role’s requirements
- You are not the final hiring decision maker; you are providing feedback to a hiring manager or committee
Your goals for the interview are:
- Allow the candidate to perform as well as they can
- Identify the candidate’s skills and ability at those skills
- Provide a clear, unambiguous hiring recommendation to the decision makers, comparing the candidates skills and abilities against those required by the role
- Provide sufficient information about the interview to allow the decision makers to disagree with your recommendation
Set aside time to prepare
You must go into the interview prepared, knowing what you expect to ask the candidate, likely detours, and expectations for their performance.
If you're not used to interviewing allow several hours for this. As you do more interviews this should take less time, but with more than 400 interviews under my belt I still allow at least an hour.
Do this several days before the interview if possible. 30 minutes before the interview is the wrong time to discover you’ve been asked to interview a candidate you’re not qualified to interview.
You should go into the interview with an expectation as to how the candidate will perform on the questions you ask them. This allows your feedback to focus on whether or not they met those expectations.
This also allows the hiring group to decide your expectations were wrong and evaluate your feedback in the context of their expectations instead.
Your expectations about a discussion on a topic with a candidate with 2 years experience and one with 10 years experience should be very different.
Read the candidate's CV. Pay attention to:
- Their most recent roles. Anything in the last three years is reasonable to expect them to be good at. Anything before then is going to get increasingly rusty.
- Specific projects they have worked on, and their role on those projects
- Specific technologies they have used
- Anything that overlaps with areas you are an expert in
If they list them, look at any open source or other public contributions the candidate has made. You're looking for things that will let you go into the interview thinking “If I ask the candidate about X then they should be able to do very well”.
Do not form an opinion if there isn't any open source work to review. You have no insight into why, and it is also inappropriate to ask the candidate. This gives you zero signal.
Setting the expectations down before you've met the candidate may also help prevent forms of bias creeping in after you've had the chance to meet them. This is a double-edged sword, of course, be careful of any biases you may have from the information in their CV.
Prepare questions to ask
You probably have a set of questions you're comfortable asking and stick to those where possible. That's ok, but don't blindly repeat the same question interview after interview.
Tailor the topics to the candidate's experience
You want the candidate to do as well as they can. So focus on playing to their strengths, while still being relevant to the role. If they do not meet the expectation you want to be able to say “I gave them every opportunity to demonstrate their abilities”.
For example, if you typically start infrastructure scaling discussions using web services as an example, but the candidate doesn't have experience with web infrastructure, but does with mail (SMTP) infrastructure, adapt your questions accordingly.
Don't: Ask questions you can predict the candidate won't do well on. For example, anything involving the minutiae of a programming language they last used 3 or more years ago doesn't make sense — they're probably too rusty on the specifics to get a useful signal.
Or if the candidate's CV suggests they have experience working with systems of ~ 10 servers, don't ask them to design something that needs ~ 1000 servers — once you go up by more than two or three orders of magnitude the scaling issues and bottlenecks become very different. Since you can already predict the candidate will probably do poorly on the question there's no point in asking it.
Don't: Get hung up on having, e.g., three topics prepared, and you want to make sure you get to discuss all three topics. If you've started on a topic and you've found yourself having an interesting, deep technical conversation with the candidate and you feel like you can continue to explore the topic and learn more about the candidate's abilities then do so.
But, if you feel you have got all the information you need from the discussion (e.g., their answer so far is going to help you make a firm yes/no decision) then you can stop early and move to the next topic.
Start writing your feedback
Once you have your understanding of the candidate, the topics you're going to cover, and your expectations, write those down as the skeleton of your draft feedback.
This helps speed up the process of submitting feedback after the interview.
It also helps guard against any bias happening during the interview. Perhaps the candidate has a particularly charming personality, and you've left the interview convinced they should be hired because of that instead of their skills. Setting out your expectations now helps ensure they are not coloured by experience during the interview.
During the interview
Keep track of time
Pay attention to the time in the interview.
You want to make sure you get through the topics you had planned to cover (but see the previous section, re being flexible).
You should note down how long the candidate spent on certain topics. How long a discussion takes to hit the points you hope to cover should be part of your expectation going into the interview.
Don't Interrupt.
Allow the candidate to think. This is especially important in a design or coding interview.
Silence during an interview while a candidate is thinking can feel like an eternity.
Resist the temptation to fill the void by talking while the candidate is thinking, designing, or coding. Similarly, if you see they've made a mistake do not point it out immediately, wait.
Talking can disrupt their flow of thought — imagine if you were trying to write code with someone looking over your shoulder interrupting every time they noticed a typo.
It also robs the candidate of the opportunity to notice and fix the mistake themselves. Seeing whether a candidate can spot a mistake on their own, and how they fix it, can be useful information.
Make sure you have told the candidate this at the start of the interview, and it's ok for them to ask questions or for help if they get stuck on something. But wait for them to ask, do not offer it unbidden.
After the interview
Finish writing up your feedback.
You need to:
- Provide a clear opinion, supported by evidence, as to whether or not you think the candidate should be hired
- Provide enough evidence so if the hiring committee decides they disagree with your opinion they have enough information to reconstruct what happened in the interview and form their own opinion.
Recommended structure of feedback
I recommend you structure your feedback as follows.
- Summary, consisting of three paragraphs.
- Explain what you understood from the candidate's CV
- Set out your general expectations
- A conclusion, explaining whether you think the candidate should be hired. If it helps, start this with the phrase “We should hire X because...” or “We should not hire X because...” Be extremely clear in explaining your conclusion, calling out specific parts of any answer you think was particularly strong or weak. If you have not come to a conclusion then you've wasted the interview
- Discussion sections
- For each topic you discussed set out:
- The initial question, as you asked it to the candidate
- Your expectations ahead of the interview
- Details from the discussions
- Your assessment of the discussion relative to your expectations
Include code and designs
If the candidate designed something in the interview, include the design. If it went through multiple iterations, include the iterations.
If the candidate wrote code during the interview include the code in your feedback. If it went through multiple iterations, include the iterations.
Be extremely clear in your feedback about what you told the candidate about what is/is not acceptable in code.
For example, you will probably get different code from a candidate if you say “I just want to see something that works, don't worry about making it testable, or checking for errors” than if you say “Please write code you would be comfortable sending for a code review”.
If you told the candidate the former, and the hiring group doesn't know, they may well err on the side of caution and assume the candidate always writes scrappy code.
If the candidate called out anything they said they would normally do, but are omitting for speed during the interview then call that out too. For example, if the candidate is writing a function that has to interact with the current time they might say something like “Ordinarily I'd pass an object in with methods to get the date/time here, because it makes this much easier to test. I'm going to skip that for the moment” then this is valuable (and good!) thinking on the part of the candidate. Note it down.
Appendix
Sample feedback
This is an example of the feedback format, and the level of detail I want to see when I'm trying to figure out if we should hire someone.
If your response to this is “Holy crap, that's a lot of detail, I'm a busy engineer, I don't have time to do this” then I'm here to tell you you're wrong. Growing the size of the team with skilled engineers is one of the largest-impact things you can do. The recruiting effort requires time from coordinators, recruiters, other interviewers, and, of course, the candidate. Don't waste their time by turning in a single paragraph of poorly detailed non-specific feedback.
Summary
Alexa has been working as a developer for 4 years, most recently implementing the design of a backend system for a crowd-funding site, growing 100x in volume over the last two years, written in Ruby. She's part of a team of 6, and appears to have been promoted up the eng. ladder once in those 4 years, which is promising.
Expectations: She should be able to talk about the project in detail and be comfortable writing code. I'm not expecting significant design skills yet due to relative lack of experience and being on a team where others are doing the bulk of the design work.
Conclusion: Hire. She communicates clearly and can talk in a detailed, organised fashion about the work she's been doing. She had sensible ideas about improving the project to scale it out. Her code was clean, and idiomatic, the tests made sense, and she didn't over-engineer things to deal with problems that don't exist. Based on this, I am confident I could give her a project like X and she would be able to complete it independently, which meets the expectations for this role.
Topic: Describe your most recent project
Expectation: She's been on this project for 4 years, 2 as a junior dev, 2 as a mid-level dev, so should have a solid understanding of it, the tradeoffs, and the areas she feels can be improved. She should also be able to explain, in detail, how the system has scaled, and failure modes that have been anticipated and designed around.
What happened:
[Include the narrative here, which I've skipped because I'm writing a blog post, not interview fanfic]
Conclusion: Good.
- She gave a detailed, organised description of the system, structured around the path a user's request takes (I thought this was a good approach, it introduced each area of the system in turn, and in a relevant context, each explanation building on information from the previous explanation).
- She understood the current scaling issues the system is facing, and the suggestions she made for fixing them are sensible
- She was able to talk about how the current system could be split up along service boundaries to re-implement services in more performant languages to deal with future scaling issues.
- Her proposed approach to testing the migration (modify frontends to send query traffic to old and new backends, compare the results for equality, export data about the result, throw away the result from the new backend) is sensible.
Topic: Write FizzBuzz in your language of choice
The problem is to, for the numbers from 1 up to some limit, print either:
- “FizzBuzz” if the number is divisible by 3 and 5
- ... or “Fizz” if the number is divisible by 3
- ... or “Buzz” if the number is divisible by 5
- ... or the number
I asked for code she would be comfortable sending for initial code review, but not to worry about documentation comments for functions. I did not specifically ask for tests or any error checking to see what she suggested.
In case it wasn't clear, this is a joke question. Do not ask this. Tom Dalling has a great blog post, “FizzBuzz in too much detail”, https://www.tomdalling.com/blog/software-design/fizzbuzz-in-too-much-detail/ which explores this rabbit hole
Expectation: She lists Ruby as her preferred language, and the project she's been working on for the last four years is Ruby-exclusive, so she should be very familiar with it. I expect initial working code in less than 10 minutes. If she doesn't suggest it, I'll ask for tests.
What happened:
She noted down the problem to refer back to it and repeated it back to me, said she'd focus on getting something that worked first, then clean it up, opened an editor, and in just over 3 minutes had written:
def fizzbuzz(limit)
1.upto(limit) do |i|
if i % 3 == 0 && i % 5 == 0
puts 'FizzBuzz'
elsif i % 3 == 0
puts 'Fizz'
elsif i % 3 == 0
puts 'Buzz'
else
puts i
end
end
end
This has a bug (she repeated the i % 3 case for Buzz). Before running the code she re-read it through and spotted the problem and corrected it without my prompting.
After fixing the code she ran it and confirmed it worked. Unprompted, she said she'd like to quickly re-write it to make this problem less likely. I agreed, and she re-wrote it to:
def fizzbuzz(limit)
fizz = (i % 3 == 0)
buzz = (i % 5 == 0)
1.upto(limit) do |i|
puts case
when fizz && buzz then 'FizzBuzz'
when fizz then 'Fizz'
when buzz then 'Buzz'
else i
end
end
end
She wondered aloud about whether this should accept more parameters to specify the text to print, or the specific divisors to use, and decided this was probably over-engineering, unless there was a business case for it. I agreed.
She said she'd want unit tests for it before it was ready for review, and I agreed. She started sketching out a test, then quickly realised the function has side effects (printing the output), so rewrote it to return the results, and a set of tests to verify the return value.
# Assume I'm sufficiently familiar with Ruby to write that code and test here.
# I'm sure you get the gist
This was the final code. We spent 25 minutes on this topic.
Conclusion:
Good:
- Most importantly: Wrote working code that solved the problem
- Wrote the problem down first, repeated it back to confirm understanding
- Focused on getting something working instead of over-optimising from the beginning
- Read the code to double check before running it, spotted a problem, fixed it
- Implemented sensible refactorings
- Suggested writing tests without prompting, as part of making it review-ready
- Good sense to not over-engineer code without reason
Could be better
- Could maybe have realised earlier that printing the results directly would make the code more difficult to test. Although as an iterative approach to development I'm fine with it.