Building An Engineering Interview Process

Creating an interview process can be tough. An effective interview process is a series of hard tradeoffs. On the one hand, you want to figure out how well a candidate will into your team and their ability to the do the job. On the other, the candidate wants to know how much they’re going to enjoy working with you and the role. Balancing these needs can be expensive, time-wise.

Spending eight hours interviewing a candidate comes with a non-trivial cost to you: that’s several hours taken away from development time, multiplied by the number of candidates you’re interviewing. That also means the candidate is spending eight hours with you, on top of any other companies they’re interviewing with and their day job. But, alas, the less time you spend, the worse the signal is for both you and the candidate. Your goal is to spend as little time as possible to get the highest signal possible. This is harder than it sounds.

Candidate experience is essential

One caution before we get into this: pay attention to the candidate experience during the interview. Even if the candidate doesn’t get an offer, you want them walking away talking about what a great team you have and how great their experience was. They may not be a good fit today, but they might be in the future. And never forget that candidates talk: if you have a crappy interview experience, other potential candidates will find out. That’s bad news for you.

So be kind. Be thoughtful. Make your interview process a delight to go through.

A Starter Process

If you’re not sure where to start with an interview process, I’ve built one out for you. This works well as-is, but please feel free to modify it to suit your own needs. Here’s the high-level:

Step 1: Job description & team prepStep 2: Screen resumes/applicationsStep 3: Phone/video screenStep 4: HomeworkStep 5: On-siteStep 6: Review feedback from interviewersStep 7: Make an offer/send the rejectionStep 8: Collect feedback from the candidate

Step 1: Job description & team prep

Hiring someone starts well before you ever post the job somewhere and start interviewing candidates. I’ll cover writing great job descriptions and how to figure out what you want in another post, but suffice it to say, your team plays a big part in determining what sort of work this new role will be doing and the kind of person that would fit well. When it comes time to start interviewing, your team needs to be crystal clear on what matters for this role and what doesn’t.

It sounds silly, but one of the most common challenges you face in hiring is not knowing what you want. “How could I post a job and not know what I want?! I wrote the job posting!” you might be thinking. Unfortunately, it’s not uncommon to have a laundry list of “desirable skills and experiences” and for those to cloud your judgment. Before long, you end up with an ideal candidate profile that is more fiction than reality. Without getting clear about what you need and what you want, you may end up rejecting many otherwise-great candidates simply because they didn’t meet your nebulous, vague idea of what you’re looking for. Every interviewer must be on the same page here.

Before you begin interviewing candidates, hold a team prep meeting. This is a great way to go over the job posting and confirm what your team wants in the new-hire. I also recommend going over some effective interview questions for assessing various skills. Each person involved with interviewing should know what they’re going to be discussing, why, and what they’re looking for. Be sure every interviewer will be taking notes during their interviews — this is vitally important to reviewing candidates later.

Step 2: Screen resumes/applications

Once your job is posted, resumes will (hopefully) start coming in. Depending on the role, your location, and a few other factors, you may have a deluge of resumes…or a trickle.

Either way, you have resumes to screen. This is a pretty quick process, though it can be tiresome and daunting, especially if you’ve been in the situation of having a few hundred resumes to sift through.

Who does the screening?

Some managers like to have an engineer on the team screen resumes, while others prefer to handle this task themselves. Either way works well (though I, personally, prefer to have managers screen them).

How do I screen resumes?

I’ve found there are two ways to screen resumes: filtering out and filtering in. Both are useful, depending on what you’re trying to hire for.

Filtering out

If you have hard technical needs, this is an effective way to screen. Review each resume, looking for those specific experiences you can’t do without and you can’t train for. Toss any that don’t meet your criteria. The rest move on to the phone screen. Easy as that.

Of course, the more hard requirements you have, the fewer resumes that make it through this step.

Filtering in

This is an effective way to screen for most roles, as it biases to the question, “Does this person have the potential ability to do this job?”

It’s much harder to screen this way since you’re not comparing against some set criteria. The upside is that many more candidates will make it through this filter.

Examples of each approach

Filter-out: Your team works on search technologies, making heavy use of Elasticsearch. You’ve recently lost a member of your team and are replacing them. You need someone already familiar with Elasticsearch and can’t spend the time to train anyone. A filter-out screening approach would toss any resume that doesn’t have Elasticsearch experience.

Filter-in: Your team works on search technologies, making heavy use of Elasticsearch. You’re adding new team members to an already stable and productive team. A filter-in screening approach wouldn’t pay much attention to Elasticsearch experience but would pay more engineering to general engineering background.

If you’re not sure which approach to use, go with the filter-in approach.

I encourage a response to all resumes at this stage, including rejections. Rejecting a candidate at this stage can be done with a form response and doesn’t need a personal response (unless the candidate came in via referral — then it should be a personal response).

Step 3: Phone/video screen

Time: 20–30m

Once you have candidates that have passed your resume screening, it’s time to talk to them. I prefer video calls instead of the phone here. With video, you both get the benefit of non-verbal communication. This call doesn’t need to be very long.

What to talk about

Ask what they’re looking for in their next role. Get to know their interests in a company and the kind of work they’re looking for. Talk about how your role maps to their interests and desires. The candidate will probably have some questions about the role as well.

You may be inclined to do a short coding test here, such as fizz buzz or another engineering puzzle/brain teaser. I recommend against it. These sorts of tests are low-signal, high-pressure, and rarely map to actual work. When was the last time you had to implement a sorting algorithm or traverse a linked list? Not since college? So don’t do it here.

Instead, ask questions about their experience. Open-ended questions are very effective: “Tell me about a recent project you worked on,” and keep asking more questions about it.

Not sure how to dig into something? Here are some questions to follow up with:

• How did you come to the decision to do X?
• What was challenging about X?
• What was the most interesting thing about X?

Remember that your goal of this interview isn’t to decide whether or not to hire them, but whether you want to continue and see how well they would fit into your team and company. Keep in mind that the candidate is probably interviewing with more than one company; you want to leave a great impression on them, so they want to continue with your company.

If you want to take them on to the next step, a quick introduction to your interview process is a good idea (phone screen, homework, on-site), as well as the typical timeline.

Lastly, ask about compensation expectations. Some candidates like to leave that discussion to the very end of the interview process, while some like to get it out of the way up front. If you know your company doesn’t pay market rate, it’s worth letting them know your budget for the role now rather than taking them through to the end of the process and finding out you’re nowhere near their price. Do not ask about their current/past compensation: not only is it now illegal in some US jurisdictions, it’s not actually helpful to you. If you really must know, just ask, “What are your compensation expectations?” They may turn around and ask about your budget — answer truthfully. There’s no need to be cagey about any of this.

As soon as the candidate has spoken with a person, you’re out of the territory of form responses. If you reject a candidate at this stage or any subsequent one, either the hiring manager or the recruiter should do so personally.

Step 4: Homework

Time: 2–3 hours (for the candidate), 5–10 hours (designing the homework), 1 hour (reviewing the homework)

Asking a candidate to do homework is a contentious point with many people, and there are great arguments for and against it. On the one hand, it is one of the strongest signals you will get from the entire interview process. On the other, many people don’t have time to do homework due to work schedules and family commitments. In essence, by choosing to provide homework, you’re making a tradeoff: in exchange for higher signal, you’re potentially filtering out entire groups of candidates.

That said, I think the benefits outweigh the downsides, and the downsides can be managed and mitigated to a large extent. One of the biggest issues with homework is the time commitment it requires, so your homework assignment must be very well time-boxed, preferably to no more than 2–3 hours.

Designing an effective but not time-consuming homework assignment is tricky. The best homework tests real-world work, so make it as close to the work they’d do for you as possible.

Most candidates will want to go above-and-beyond, spending way more time than you expect. On top of that, I’ve seen several homework assignments that went untested entirely, or were only tested by the person who wrote it. The best test: pay a couple people to go through it and provide feedback on it. For a few hundred dollars, you can be sure your test is both effective and not time-intensive.

A caution

Some teams like to have the candidate write an actual feature or fix an actual bug. While that is as close to real work as you’re going to get, it has two enormous problems:

1. It requires more context and knowledge of your app than they have, so you are unlikely to see them at their best.
2. It is valuable to you on its own. That is, you benefit from the task even if you don’t hire them. This amounts to free work. Not cool.

If a candidate submits homework, someone should review it. Rejecting a candidate without reviewing their homework assignment is disrespectful of the candidate’s effort. If your homework takes too much time to review, consider redesigning it, so the review is easier.

Step 5: On-site

Time: 4.5 hours

The on-site is where you finally get the candidate and your team together and really understand how well they can do the job and mesh with your existing staff. Don’t forget that the candidate is also interviewing you here. This is not one-sided — leave room for the candidate to get to know you and your company in every interview.

I’ve put together a sample schedule below. Some notes about this schedule:

• Modify it to suit your needs! This is only a sample.
• Pay attention to candidate experience. The small things can have a big impact. For example, have a couple bottles of water in every interview room. Restock as necessary; your candidates will be thankful for it.
• Send the schedule to the candidate a couple days before the interview, so they know what to expect. Even better if you can tell them ahead of time who they’ll be speaking with, so they have time to look them up on LinkedIn/Twitter/Google. It’s much easier to build rapport when you can establish common ground beforehand.
• If you schedule a lunch, be sure to ask the candidate about any dietary restrictions when you send the schedule. You really don’t want to take a candidate to a lunch they can’t eat.

Some special notes about lunch:

Rule #1: no alcohol. Period. End of discussion. That goes for your team as well. Alcohol during an interview will signal the worst of tech culture to the candidate, and if it happens that the candidate doesn’t drink alcohol, they may feel awkward — especially if your team is all drinking and they’re not.

As for who to go to lunch with, pick two of the previous interviewers. If possible, not the manager or recruiter. Going to lunch with the manager is too high-pressure since they are perceived as having the final say on whom to hire (where that’s actually true or not for your company is irrelevant). Going to lunch with the recruiter isn’t helpful to the candidate since they’re not going to be working with the recruiter.

So pick two of your team to go to lunch with them. Why two? If it’s just one team member and the candidate, it’s possible that there’s little rapport with that specific person and awkward pauses are too common. More than two and it’s easy to fall into the trap of your team talking to each other and leaving the candidate out, not to mention it can be a bit overbearing having so many people. The sweet spot is two people, as the social dynamics will allow for conversation to flow more freely.

Sample On-site Schedule

Sample on-site schedule

Step 6: Review feedback from interviewers

Within a couple of days following the on-site, all feedback needs to be reviewed. I recommend doing this as a group so you can discuss the feedback. This is where the value of the note-taking comes into play: if you don’t have actual notes to review, then you’re only able to discuss instincts and gut feelings, which we know aren’t accurate. Without notes, you can’t suitably review a candidate.

Step 7: Make an offer/send the rejection

Offer

Oh, happy day — you found a candidate you want to hire! But you’re not done yet — the offer hasn’t been accepted, and you haven’t really made a hire until the candidate is sitting at one of your desks, working for you.

Making an offer is straightforward and should come in two conversations:

The first conversation is to call the candidate and let them the company would like to extend an offer. Now it’s time to talk real details: salary, non-cash compensation, leave time, benefits, etc. Learn what is important to the candidate — not everyone cares about maximizing salary.

The second conversation is the verbal offer itself. Call the candidate and let them know the details of their offer. After the call, send them the offer letter via email, along with all details of the benefits package.

Some companies have what’s known as an “exploding offer,” which is an offer with a deadline attached to it. Please don’t do this. It’s unfair to the candidate who might need time to wrap up other interviews or discuss it with their family. Instead, ask about their expected timeline for an offer. No one is going to sit on an offer for months and expect it to be around still, so don’t worry too much about it. Just keep communication open with the candidate, and it’ll be fine.

After the candidate has spent time reviewing the offer, some negotiation may be needed. If everything is great for both sides, then you end up with an accepted offer. Congratulations!

Rejection

Rejection at this stage is hard on the candidate. At this point, if you’ve used my schedule, the candidate has potentially spent eight hours with you, potentially more if they put additional time into the homework. They’re invested at this point.

While you may casually reject a candidate, know that it is anything but casual for them. Any rejection deserves a personal response from the hiring manager. A vague rejection (“Sorry, the team has decided it’s not a good fit at this time”) is rough, especially if the candidate felt the interviews had gone well. Have empathy for their position.

If the basis for the rejection was technical in nature, you should tell the candidate so and make recommendations about how they can improve. If the rejection was non-technical, that’s a bit touchier. Your HR and Legal teams will likely have a complete fit over this — I recommend asking them to walk you through what the risks are. Just remember, a vague rejection is rough and unfair to the candidate. Keep that in mind.

Step 8: Collect feedback from the candidate

After the entire process is complete, regardless of whether you made an offer or rejected them, you should collect feedback from the candidate. This can take the form of a few questions over email or even a Google Form. The feedback from the candidate is valuable information for improving your interview processes, spotting interviewers that need additional training, and much more. If you’re sending this to someone, you made an offer to, wait after the offer is accepted and make sure the candidate knows it won’t have any impact on their offer.

So there you have it: a starter interview process. What do you think? How does your own process work? How well does it work for you?