Did they pass the Coding Exercise?

This is more or less another observation on how something is “a means to an end”… but I think there is a fun analogy here staring us right in the face: our journey starts with the word exercise.

Why is it called a coding “exercise”?

Any kind of exercises, whether they be weight-lifting, calisthenics, running, etc, are intended to guage the fitness of someone in a given context.

When you’re working out, you’re interrogating a set of muscles with resistance and seeing how they all respond in concert.

Muscular strength, then, is kind of like experience that is stored in the body.

Similarly, when you’re interviewing a candidate, you’re interrogating a set of their skills with a problem and seeing how they use their experience in concert.

Provide some kind of stimulus and observe the response.

Moving away from a pass/fail mentality

We can’t reasonably be so binary in our assessment when it comes to interviewing candidates. If only it were so simple.

For many reasons you can’t just ask someone “can you lift 500 kilos?”

But more importantly, what you’re trying to learn shouldn’t necessarily be “can they lift 500 kilos?”

What we’d like to get to is a holistic understanding of the candidate’s strengths and weaknesses.

And then we can discuss how those traits fit into the fabric of the existing team.

Validating success using indirection

A coding exercise should seek to tease out those traits without needing to take the candidate at their word.

To blend our analogy a bit, the assessments should be more in the spirit of:

• What was the quality of the reps?
  How many projects did they do and how difficult or diverse were those projects?

• How was their form?
  Did they write clean code that other engineers could reason about?

• Did they use their full range of motion?
  What kind of experiences did they draw upon and apply in a new context?

• Did they have good mind-muscle connection?
  What level of self-awareness did they have?

• Did they use you as a spotter?
  Were they receptive or eliciting feedback and suggestions?

Now we have a rich collection of observations from which to draw upon after the interview.

We can now have an idea of which level the candidate operates at and also what kind of potential they might have.

And even more importantly, we begin to see which traits of the candidate that are inherent (and not likely to change), and which traits are coachable.

Forming your own opinion

It’s possible for a candidate to gain experience in a lot of different ways and it’s our job to glean meaningful information from all the clues.

It doesn’t mean: how many jobs did they have, and how many years did they work at each of those jobs.

It does mean: what did they learn and how did they grow while they worked at those jobs?

So given that we are interviewing them, what might all of this say about how they assimilate, grow, and raise-the-bar if they join our team?

Post-huddle

Finally, we must cross-reference our findings with the rest of the team.

Regrouping after the interview loop we will be asking ourselves, “did they pass the coding exercise?”

And we will answer with a set of qualitative observations instead of the binary “yes or no”.

And we will use those observations to decide whether the rewards of hiring the candidate outweigh the possible risks.

Since hiring is an extremely expensive process (in terms of time, energy, money, everything), derisking each new hire is immensely important.¹

If the only thing your team learns from the coding exercise was that the candidate could or couldn’t solve what is ultimately a contrived or academic problem, then you are probably going to miss out on the chance to build a really great team.