Being an Effective Interviewer

One challenging aspect of being an engineer is interviewing other engineers. The interview process is rife with various problems, including the discomfort of interviewing someone who you perceive as being a better engineer than you are, or figuring out how to draw out actual engineering skill versus simply finding out how much someone has memorized. Does that CCIE or degree on their resume really mean anything? What does an effective network engineering interview look like?

There are, of course, several different theories “out there.” For instance, some companies focus on giving candidates “real world” problems to solve, and checking with them several days later to see how they’ve done. This is potentially useful, but quite often I find my best work is done with a team, rather than by myself. Such systems seem to tend towards pitting the candidate against well known or well established problem sets, which can easily revert back to memorization skills, or towards difficult/obtuse problems. Either way, this doesn’t test the candidate’s ability to work in a team, or interact with others in solving difficult problems.

What about tossing other sorts of puzzles towards the candidate to see how they do? This also seems problematic to me in many ways; some people don’t do well on puzzles while under pressure to perform, for instance, so the process may result in hiring a lot of people who perform well under pressure, but, again, don’t know how to work on a team, or are lacking in basic “human qualities,” like ethics (now there’s a word you’ve not heard in a long while in the engineering world, I’ll bet).

So what is the solution here? I actually don’t think there is a single solution to this problem—but I do have some interviewing techniques that might be helpful. What I’m after is allowing the candidate to succeed. I’m not going to learn anything by giving someone a test I know they’ll fail, and then walking away. I might improve my ego somewhat, but that’s not going to help me, or my team, actually get a job done—and if my ego is the most important thing in the interview, I’m probably not doing a good job for the team anyway. But I want to allow the candidate to succeed in a way that allows me to see how low the bar must be taken for them to succeed, or, alternately, by starting with a low bar and inching it up until they run out of engineering steam.

How does this work? There are two basic techniques.

First, I start with some project the candidate has worked on in the past. For this project, I begin by asking what they did, looking for answers around how they deployed and configured specific devices, protocols, or solutions. If those answers make sense, then I move to why did you do this? Here I would expect an initial set of answers around application support, technology limitations/operation, and etc. At this point, though, I can drive in two different directions—and often I can go both directions at the same time. The first is asking what business drivers pushed this particular decision; the candidate has the chance to show some business integration knowledge at this point. The second is in asking why the particular technology limitations exist, which should result in some discussion around the way the technology works, including the tradeoffs made in the technology design.

Using this process it’s pretty easy to tell where the candidate has “stopped thinking,” and has therefore “topped” in their ability to understand technology.

Second, if the situation calls for it, I set up a simple problem, and add complexity to it over time. A common one is to start with two hosts and a single router drawn on a white board. Assume all three are turned off, and they are all turned on at the same time. Now—explain precisely the process one of the two hosts uses to communicate to the other host. This is very basic, but that is the point—give them a problem you’re fairly certain they should be able to solve to set the stage and make them comfortable.

Once the initial question is answered, you can “ramp the problem up.” For instance, add a second router, and ask how the two routers should learn about connected interfaces. Add an attacker, and ask what techniques might be used to protect the hosts, and the communications between them. You can ramp this type of problem up quickly or slowly until you think you’ve reached either the end of your time, or the end of the candidate’s ability to go farther.

Neither of these ideas are perfect, of course—they are just two of the processes I’ve learned over the years to try and understand the skill level of another engineer through the interview process. And they both seem to work (at least for me) with candidates of all skill levels, and without either pushing my ego to the forefront of the process or failing to get any substantial information out of the interview.

If you have other ideas, or techniques, you’ve used over the years, feel free to post them in the comments.