Why you should hold auditions and stop doing interviews.

Job interviews don’t actually work. The evidence is unequivocal.

There are thousands of ways to do them, they are all terrible.

“Tell me about a time when you…”

“Solve this rubix cube…”

“Do this psych test….”

I must have done hundreds of interviews over my career in tech and here’s the rub.

There may occasionally be a correlation between how well someone does in an interview and how well they perform on a job, but it’s absolutely not a predictor of future performance.

Most of the time you just hire the best actor on the day.

If you want to see just how far you can go with puzzle style interviews, there is a great book called ‘How would you move Mount Fuji’ about how Microsoft used a series of intellectual puzzles to hire people, only to learn years later, what I’m telling you now — it’s a total waste of time.

So what can you do?

A few years back, I heard about this study where they take a bunch of students in a room and have a teacher come in to give a class. After 5 minutes, they take the teacher out, and have the students predict how good of an educator the teacher will be.

Next, you do the same experiment, but you have the teacher teach an entire semester. At the end of the semester, you give the students the same questions and low and behold, what you find is that on average the students will give the same scores to the teacher. The amount of time someone spends with someone doesn’t appear to alter their perceptions in how good of a performer that person will be in the future.

Essentially, people make up their minds about how good someone will be after about 5 minutes.

What’s the alternative?

After hearing about this study, and listening to Matt Mullenweg from Automattic talk about auditions, something just clicked. I realised I had wasted so much time.

The idea of doing auditions is simple. Here is how I do it (and its different to how it works at Automattic).

Bring someone in, have a chat with them for a few minutes to get through some of the basics. How much money do they want, when can they start, what are they wanting to do, that kind of thing.

Then, we bring them for a day, or a few hours, and give them an actual thing we’re working on, a problem we’re trying to solve. We sit them down at a desk if they are in the office, or we tell them to work from home if they are going to be remote. We then work on a project as if they already had the job.

Its a good way for them to see if they like working with us, and see if we like working with them. You see what they grasp and where they struggle.

Its a little like how you might hire a new chef. You bring them in, talk for a while, then you ask them to make an apple pie. You get to see how they work, do they work clean? Are they fast? In the end, what matters most is how the food tastes.

Auditions are about the food, not the knives.

The problem with interviews is they are about everything but the food.

Its about how well you dress. How well you charm the room. How early in the day it is and how tired everyone is. I know if I personally do interviews at the end of the day when I’m tired, I’m probably much harder on candidates than I would be if I did it first thing in the morning when I’m nice and fresh.

The way to see if someone will be good at a job is to, well, as obvious as it sounds, see if they are good at the job. The way to do that is through auditions, not puzzles.

What are the upsides?

I’ve probably done 30 auditions now and there is a couple of things I’ve found that I thought I’d share.

• It’s much faster. Which is better for you and the candidates.
• I have heard from probably 40% of the candidates that they think the process is refreshing. They see it’s honest and it removes the bias. Generally it’s just less bullshit and people respond to that.
• It means it doesn’t really matter what school you went to, what your background is or who your friends are, it only matters that you can do the job.
• Candidates like this process better.
• It means being able to write a flashy resume doesn’t guarantee you from getting a job that has nothing to do with writing flashy resumes. That’s a good thing.
• Staff like the process. It wastes less of their time.
• It encourages more diversity as it removes more bias from the process, especially with remote staff.
• It allows for blind comparisons of the outputs. If you compare all the work done in the audition and remove the names from the work and have other staff compare the results, its a great way of ensuring you’re hiring the best candidate.

Challenges?

• You need to understand the difference between Skills and Competencies and take that into account. You can’t expect people to know internal procedures for example, so don’t evaluate that. Don’t expect them to know how you do things. Let them do the audition their way. Adding too many guidelines can make it confusing.
• The tasks can only be kind of small, unless you do what they do at Automattic does and actually hire people for a few weeks for an extended audition.