Nik's guide to interviewing

Because life's too short to spend it reading terrible interview feedback

This is my rough guide to how to prepare to do good tech interviews and provide useful feedback. It’s not a one-size-fits-all guide; it’s the best practices I’ve adopted after giving more than 400 technical interviews to candidates with all ranges of experience and backgrounds who are applying for roles with a strong software engineering component to them.

It assumes you're an interviewer using the typical tech interview format of a 45-60 minute discussion with a candidate, generally focusing on a specific area of knowledge.

I'm not saying that’s the right way to conduct candidate interviews, or commenting on the various merits of this approach vs. take-home coding interviews, or pair-programming with an interviewer, or any other approach. But if this is the approach your organisation has taken to interviews, this is how you can conduct them well.

Before the interview

I assume:

Your goals for the interview are:

Set aside time to prepare

You must go into the interview prepared, knowing what you expect to ask the candidate, likely detours, and expectations for their performance.

If you're not used to interviewing allow several hours for this. As you do more interviews this should take less time, but with more than 400 interviews under my belt I still allow at least an hour.

Do this several days before the interview if possible. 30 minutes before the interview is the wrong time to discover you’ve been asked to interview a candidate you’re not qualified to interview.

Form an expectation

You should go into the interview with an expectation as to how the candidate will perform on the questions you ask them. This allows your feedback to focus on whether or not they met those expectations.

This also allows the hiring group to decide your expectations were wrong and evaluate your feedback in the context of their expectations instead.

Your expectations about a discussion on a topic with a candidate with 2 years experience and one with 10 years experience should be very different.

Read the candidate's CV. Pay attention to:

If they list them, look at any open source or other public contributions the candidate has made. You're looking for things that will let you go into the interview thinking “If I ask the candidate about X then they should be able to do very well”.

Do not form an opinion if there isn't any open source work to review. You have no insight into why, and it is also inappropriate to ask the candidate. This gives you zero signal.

Setting the expectations down before you've met the candidate may also help prevent forms of bias creeping in after you've had the chance to meet them. This is a double-edged sword, of course, be careful of any biases you may have from the information in their CV.

Prepare questions to ask

You probably have a set of questions you're comfortable asking and stick to those where possible. That's ok, but don't blindly repeat the same question interview after interview.

Tailor the topics to the candidate's experience

You want the candidate to do as well as they can. So focus on playing to their strengths, while still being relevant to the role. If they do not meet the expectation you want to be able to say “I gave them every opportunity to demonstrate their abilities”.

For example, if you typically start infrastructure scaling discussions using web services as an example, but the candidate doesn't have experience with web infrastructure, but does with mail (SMTP) infrastructure, adapt your questions accordingly.

Don't: Ask questions you can predict the candidate won't do well on. For example, anything involving the minutiae of a programming language they last used 3 or more years ago doesn't make sense — they're probably too rusty on the specifics to get a useful signal.

Or if the candidate's CV suggests they have experience working with systems of ~ 10 servers, don't ask them to design something that needs ~ 1000 servers — once you go up by more than two or three orders of magnitude the scaling issues and bottlenecks become very different. Since you can already predict the candidate will probably do poorly on the question there's no point in asking it.

Don't: Get hung up on having, e.g., three topics prepared, and you want to make sure you get to discuss all three topics. If you've started on a topic and you've found yourself having an interesting, deep technical conversation with the candidate and you feel like you can continue to explore the topic and learn more about the candidate's abilities then do so.

But, if you feel you have got all the information you need from the discussion (e.g., their answer so far is going to help you make a firm yes/no decision) then you can stop early and move to the next topic.

Start writing your feedback

Once you have your understanding of the candidate, the topics you're going to cover, and your expectations, write those down as the skeleton of your draft feedback.

This helps speed up the process of submitting feedback after the interview.

It also helps guard against any bias happening during the interview. Perhaps the candidate has a particularly charming personality, and you've left the interview convinced they should be hired because of that instead of their skills. Setting out your expectations now helps ensure they are not coloured by experience during the interview.

During the interview

Keep track of time

Pay attention to the time in the interview.

You want to make sure you get through the topics you had planned to cover (but see the previous section, re being flexible).

You should note down how long the candidate spent on certain topics. How long a discussion takes to hit the points you hope to cover should be part of your expectation going into the interview.

Don't Interrupt.

Allow the candidate to think. This is especially important in a design or coding interview.

Silence during an interview while a candidate is thinking can feel like an eternity.

Resist the temptation to fill the void by talking while the candidate is thinking, designing, or coding. Similarly, if you see they've made a mistake do not point it out immediately, wait.

Talking can disrupt their flow of thought — imagine if you were trying to write code with someone looking over your shoulder interrupting every time they noticed a typo.

It also robs the candidate of the opportunity to notice and fix the mistake themselves. Seeing whether a candidate can spot a mistake on their own, and how they fix it, can be useful information.

Make sure you have told the candidate this at the start of the interview, and it's ok for them to ask questions or for help if they get stuck on something. But wait for them to ask, do not offer it unbidden.

After the interview

Finish writing up your feedback.

You need to:

  1. Provide a clear opinion, supported by evidence, as to whether or not you think the candidate should be hired
  2. Provide enough evidence so if the hiring committee decides they disagree with your opinion they have enough information to reconstruct what happened in the interview and form their own opinion.

I recommend you structure your feedback as follows.

Include code and designs

If the candidate designed something in the interview, include the design. If it went through multiple iterations, include the iterations.

If the candidate wrote code during the interview include the code in your feedback. If it went through multiple iterations, include the iterations.

Be extremely clear in your feedback about what you told the candidate about what is/is not acceptable in code.

For example, you will probably get different code from a candidate if you say “I just want to see something that works, don't worry about making it testable, or checking for errors” than if you say “Please write code you would be comfortable sending for a code review”.

If you told the candidate the former, and the hiring group doesn't know, they may well err on the side of caution and assume the candidate always writes scrappy code.

If the candidate called out anything they said they would normally do, but are omitting for speed during the interview then call that out too. For example, if the candidate is writing a function that has to interact with the current time they might say something like “Ordinarily I'd pass an object in with methods to get the date/time here, because it makes this much easier to test. I'm going to skip that for the moment” then this is valuable (and good!) thinking on the part of the candidate. Note it down.

Appendix

Sample feedback

This is an example of the feedback format, and the level of detail I want to see when I'm trying to figure out if we should hire someone.

If your response to this is “Holy crap, that's a lot of detail, I'm a busy engineer, I don't have time to do this” then I'm here to tell you you're wrong. Growing the size of the team with skilled engineers is one of the largest-impact things you can do. The recruiting effort requires time from coordinators, recruiters, other interviewers, and, of course, the candidate. Don't waste their time by turning in a single paragraph of poorly detailed non-specific feedback.

Summary

Alexa has been working as a developer for 4 years, most recently implementing the design of a backend system for a crowd-funding site, growing 100x in volume over the last two years, written in Ruby. She's part of a team of 6, and appears to have been promoted up the eng. ladder once in those 4 years, which is promising.

Expectations: She should be able to talk about the project in detail and be comfortable writing code. I'm not expecting significant design skills yet due to relative lack of experience and being on a team where others are doing the bulk of the design work.

Conclusion: Hire. She communicates clearly and can talk in a detailed, organised fashion about the work she's been doing. She had sensible ideas about improving the project to scale it out. Her code was clean, and idiomatic, the tests made sense, and she didn't over-engineer things to deal with problems that don't exist. Based on this, I am confident I could give her a project like X and she would be able to complete it independently, which meets the expectations for this role.

Topic: Describe your most recent project

Expectation: She's been on this project for 4 years, 2 as a junior dev, 2 as a mid-level dev, so should have a solid understanding of it, the tradeoffs, and the areas she feels can be improved. She should also be able to explain, in detail, how the system has scaled, and failure modes that have been anticipated and designed around.

What happened:

[Include the narrative here, which I've skipped because I'm writing a blog post, not interview fanfic]

Conclusion: Good.

Topic: Write FizzBuzz in your language of choice

The problem is to, for the numbers from 1 up to some limit, print either:

I asked for code she would be comfortable sending for initial code review, but not to worry about documentation comments for functions. I did not specifically ask for tests or any error checking to see what she suggested.

In case it wasn't clear, this is a joke question. Do not ask this. Tom Dalling has a great blog post, “FizzBuzz in too much detail”, https://www.tomdalling.com/blog/software-design/fizzbuzz-in-too-much-detail/ which explores this rabbit hole

Expectation: She lists Ruby as her preferred language, and the project she's been working on for the last four years is Ruby-exclusive, so she should be very familiar with it. I expect initial working code in less than 10 minutes. If she doesn't suggest it, I'll ask for tests.

What happened:

She noted down the problem to refer back to it and repeated it back to me, said she'd focus on getting something that worked first, then clean it up, opened an editor, and in just over 3 minutes had written:

def fizzbuzz(limit)
  1.upto(limit) do |i|
    if i % 3 == 0 && i % 5 == 0
      puts 'FizzBuzz'
    elsif i % 3 == 0
      puts 'Fizz'
    elsif i % 3 == 0
      puts 'Buzz'
    else
      puts i
    end
  end
end

This has a bug (she repeated the i % 3 case for Buzz). Before running the code she re-read it through and spotted the problem and corrected it without my prompting.

After fixing the code she ran it and confirmed it worked. Unprompted, she said she'd like to quickly re-write it to make this problem less likely. I agreed, and she re-wrote it to:

def fizzbuzz(limit)
  fizz = (i % 3 == 0)
  buzz = (i % 5 == 0)
  1.upto(limit) do |i|
    puts case
      when fizz && buzz then 'FizzBuzz'
      when fizz then 'Fizz'
      when buzz then 'Buzz'
      else i
    end
  end
end

She wondered aloud about whether this should accept more parameters to specify the text to print, or the specific divisors to use, and decided this was probably over-engineering, unless there was a business case for it. I agreed.

She said she'd want unit tests for it before it was ready for review, and I agreed. She started sketching out a test, then quickly realised the function has side effects (printing the output), so rewrote it to return the results, and a set of tests to verify the return value.

# Assume I'm sufficiently familiar with Ruby to write that code and test here.
# I'm sure you get the gist

This was the final code. We spent 25 minutes on this topic.

Conclusion:

Good:

Could be better