The Winchester job interview theory

Royal Winchester is a very smart guy. He also has a disturbing habit of thinking very well about things. He recently offered me this theory on interviews.

The theory goes as follows: Interviewing is mostly bullshit.

As the theory goes, most of us make instinctive judgments on factors we don’t understand in the first five minutes, and spend the rest of the time, and the time discussing with other interviewers, back-filling logical reasons to support an intuitive response we’re largely in denial of.

Not everyone does this of course, and not all the time, but the theory suggests it’s true much of the time, or is a significant factor in interviews.

There are some contributing hypotheses to the theory:

Few are mature enough to sort out their biases. Very few people posses the self awareness to realize why they instinctively like or do not like someone they’ve just met. And even fewer, especially among the business/engineering crowd, feel comfortable with their feelings. It’s considered unacceptable to say ‘the guy did well but I didn’t like him for reasons I can’t explain’. It’s much easier to hide that feeling inside unfair judgments, using whatever flavor of corporate jargon can be found in the official hiring criteria (Lacked intellectual horsepower, couldn’t deal with ambiguity, didn’t know the secret handshake, etc.)
Talking about doing is not doing.� Most interviewers� focus on trying to extract a prediction about someone’s ability through having them talk about their ability. This is ridiculous. Could you evaluate an NFL running back by asking them questions about how they run? (e.g. “I run really really fast”, “Great, your hired.”). Better interviewers work hard to put candidates in problems and situations like the real ones they’ll face, and watch. They collaborate on real problems during the interview, as that’s what much of work is. Over time they’re able to calibrate what it means for a candidate to do well, given real problems, in an hour. But this requires� skill and patience few interviewers have. And even when they do, the candidates are in an awkward and artificially stressful environment that does not approximate real work well, unless the interviewer is diligent on compensating for these issues.
Interviews work better as a filter.� The job interview loop is more effective at eliminating bad candidates than identifying good ones. The bet is by the end only good candidates remain, but that’s not true. Like bacteria responding to antibiotics, strains of bad candidates that are immune to your process survive as well, and are hard to distinguish from good ones. The process can be prone to false negatives too (people who get rejected but would have thrived).
Recommendations are underestimated. Since interviews are mostly bullshit, it makes sense to put more weight on a recommendation from a trusted person (not necessarily the names on the candidates resume) who has worked with the candidate somewhere else. They have first hand experience on the millions of things that can only be witnessed outside of the interview room. If you trust them, and they trust the candidate, that may have more predictive ability than 60 awkward minutes in your office.
No one else saw what happened. Interviewers are free to lie and distort, intentionally or not. All interviewers are free to invent pet theories on which questions work best, or how good they are an extracting the value of a candidate. They are the only record of what they asked, how they asked it, and how the candidate performed. If they have bad habits that bias the candidate, no one will ever know, as the candidate has almost no ability to report on the interviewer. Every interview is a cat and mouse trapped in the room, and the mouse is motivated to do whatever it can to survive the cat, no matter how cruel or unfair the cat is.
We never go back a year later and evaluate. The hiring loop at nearly all companies is broken, as there is no feedback loop. No one forces you to go back 6 months or 2 years later and see: how many of the hire decisions you made worked out well, and how many of the people you rejected kicked ass at other companies with similiar cultures and needs. With no data, the value of any interview process is guesswork, not rigor.

Despite my affinity for this theory, I believe groups that take interviewing seriously, and leaders who reward interviewers for putting more time and careful thought into interviews, end up with better teams. The choice to hire someone is the most important decision you make that month, or year and the wise know this. At a start-up it can make or break the company.� And the more seriously people take the process, even if it’s flawed, the higher the odds they’ll recognize the natural shortcomings above and invest in minimizing them.

While I don’t think all interviews are a crapshoot, I agree with the Winchester theory – in most interviews, most of the time, it’s mostly bullshit as a tool in truly evaluating how well a person would perform in the job, even if the people doing the interviews don’t intend it to be.

The real story is in who you recruit to interview in the first place. Better candidates in, better candidates out:� see my essay how to interview and hire people.

What is your take on interviewing? How do you work against the Winchester theory?

Posted Monday, January 25th, 2010 at 12:14 pm

in Management | 19 Comments » |

Recent Comments

How to write a book – the short honest truth
Johan says..."Very useful in the way that you point out some very basic pros and cons about writing, the article points out some stuff that to...'
On God and Integrity
Ricardo Duarte says..."I feel the same when I see priests praying that the soldiers of his country win a war. It is unfair, inhumane and does not represent what God truly is....'
How to write a book – the short honest truth
SHAWNTEL PERKINS says..."SORRY IT’S 431-7788 THANKS'
How to write a book – the short honest truth
SHAWNTEL PERKINS says..."HEY I’M TRYING TO WRITE A BOOK CAN YOU HELP ME PLEASE THANKS YOU CAN CONTACT ME AT MY EMAIL ADDRESS OR THIS NUMBER...'

Jason Donnell - January 25, 2010 at 1:22 pm #

I’ve had that theory for years now (including four or five of the contributing hypotheses noted). I didn’t realize I should have been letting you know! :)

Steven Levy - January 25, 2010 at 1:44 pm #

As a department head, I tried to get back with the individual interviewers six months to a year later on the occasional less-than-stellar hire to figure out what we missed. However, I was less than rigorous about this, which was a mistakes, as you point out. I was probably most rigorous with myself, since the less-than-good hire was affecting my success personally through my team.

There were a handful of misfires, but I feel good about most of the people I hired over the years. I was able to get recommendations on most of them, and I had a couple of people on my team who had really good sensitivity to something “off” about a candidate, which helped.

These days it’s very hard to get recommendations from former co-workers outside a company. In this litigious society, many companies will not allow employees to speak about former employees except to confirm dates of service.

For what it’s worth, 20 years ago, I gave a good recommendation to another company about someone I’d worked with, and the person promptly turkeyed-out on the new job, an incident I’ve never forgotten… though I’ve never been sure quite what lesson to draw from it. One thing is that I was always up front with someone from another department asking about an employee on my team; I never tried to “shop a problem” by concealing something about an employee I was unhappy about.

My-Project-Management-Expert.com - January 25, 2010 at 1:50 pm #

I think most people have no clear idea of how to interview someone effectively. Even those using a point system and psychometric testing have little idea of what kind of person they actually need or want.

In my area of project management this is even more of a problem since you can’t give someone a technical test, and qualifications are not widespread. So many of my peers are asked questions about the project management basics or project manager duties none of which tells you much.

Luckily I’ve reached a level where I’m usually only asked one question, “can you turnaround and deliver this failing project?”. But it begs the question why so many projects are in failure mode when apparentely the project manager passed the interview with flying colours!

Regards

Susan de Sousa
Site Editor http://www.my-project-management-expert.com

Scott Berkun - January 25, 2010 at 1:50 pm #

Steven: I’m convinced there are too many variables to ever isolate why one person, in one job, does great or fails.

A personal life crisis can arise for a hundred reasons that have nothing to do with their abilities, but destroy their performance.

Or a a personality conflict can demoralize someone sufficiently that they appear to be a turkey, when they’re really an eagle, or a hawk, or whatever the opposite of a turkey is.

But the fact that you can’t isolate the causes, good or bad, doesn’t suggest they should be ignored. Even though there is no answer, the teams that are rewarded for thinking about it, and investing energy in it, will, over time do better.

Justin Warren - January 25, 2010 at 3:31 pm #

I’d like to see more companies fix bad hiring quicker. Get rid of the lemons! Help them leave, find them another job they’d be better at, performance manage them out; whatever works, for goodness’ sake make it happen!

And when you strike it lucky with someone good, do what you can to hang onto them.

This combined should increase the quality of people you have over time, and hopefully these better people will find more better people more often.

Scott Berkun - January 25, 2010 at 5:53 pm #

Someone mentioned in email it’s good to filter out people based on their chemistry with the team, or with you, and I agree.

The problem is it’s hard for most people to discern between positive reasons they like someone, and negative (e.g. they share your bias or bad habits or something). We just mostly sense we like them, or they agree with us.

Simon Fairbairn - January 25, 2010 at 6:14 pm #

I once had an interview where I figured out very quickly that the interviewer loved to talk about himself.

I answered every question he asked with “Yeah, I’ve had experience with XYZ. I’ve done this with it and, as I’m sure you know yourself, found X to be very interesting…” or

This was enough to trigger another 30 minute diatribe on why he was so awesome at X. I spent the whole time smiling and nodding and laughing at the appropriate moments.

At the end of the interview, he said “I think you’re going to fit in perfectly here” and I got the job.

Carolyn - January 25, 2010 at 6:35 pm #

This is actually very encouraging for me, because I freak out from the pressure on technical questions in interviews, but I think I’m pretty good at making a good first impression. So, right on!

Jamie Irvine - January 26, 2010 at 9:43 am #

It is frustrating that with so many professional psychometric tests on the marketplace companies are still using subjective interview processes.

The formula is simple:

Step 1 – Define Your Target – What will high performance in the position you are filling look like?

Step 2 – Define Your Potential – Which candidate fits the high performance requirements?

Step 3 – Teach the manager that the candidate will report directly to how to leverage this information, removing all obstacles to high performance, and watch the individual excel.

Using a psychometric test that is accompanied by exert interpretation will help you to follow this 3 step process that leads to success.

PM Hut - January 26, 2010 at 3:43 pm #

I don’t think a lot of people work against the Winchester theory (this is the first time I hear about it anyway).

A few months ago we setup interviews to hire 2 programmers. We were 2 persons doing the interview and if any of us feels (note the word “feel”) that the candidate will not fit then we’ll thank him/her for the time, and that’s it. What’s interesting is that we ended hiring up someone, but that person elected, at the last minute, to work somewhere else, so we tried with the second best, whom I thought would not fit into the company’s culture and dismissed earlier. What’s interesting is that against all my expectations/hunches/expectations, that person thrived in the company.

Sometimes giving someone a chance, even if against your bias, might work. This is definitely an exception to the rule IMO, but again, I might be wrong.

PS: I did publish a collection of 300 PM interview question in case someone is conducting a PM interview.

PM Hut - January 26, 2010 at 3:47 pm #

Quick note, I just read Scott’s comment about the chemistry between the candidate, the manager, and the team. Well in the real life example I gave above, that chemistry did not exist during the interview (in fact I thought that the candidate was a bit rude, and I was wrong). It is important to note that during interviews a lot of candidates are not really themselves, they might lie in ways that might help them get the job (and later hurt them while in the job) or vice versa.

This is a good article, thanks.

dspman - January 27, 2010 at 10:47 am #

I really enjoyed this article, which made me recall an incredibly exhilarating interview process I went through once. I am an engineer, and went to interview for a new position. I spent the whole day in a conference room with various engineers from the company coming in and asking me how I would solve real world problems they were facing or had faced in the past. As soon as it appeared that I already knew the answer to any specific problem, they changed to something new. They were genuinely interested in how well could I think and thence express my ideas. I have to disclose that it was for a consulting position, so they needed to make sure that they would not be embarrassed by something dumb I might say in front of the clients, so that needs to be factored in. I got the job and had a great experience working closely with many of the people who interviewed me that first day.

Lynn - January 28, 2010 at 7:40 pm #

This theory would be more useful if it were accompanied by a better idea for sorting through people to determine who to hire.

In lieu of a better system, even the gut-feeling judgments most interviews result in beat the alternative – hiring someone blindly.

Mike Nitabach - January 29, 2010 at 11:47 am #

Very interesting post. I do a lot of interviews of potential post-doctoral science trainees to join my lab.

First I interview the candidate one-on-one for about an hour to see if they are capable of thinking on their feet while discussing science. Then they deliver a formal “job talk” seminar to the entire lab in which they describe their PhD research. I and the other trainees in the lab ask lots of tough questions during the seminar to try to knock the candidate off message and see how they respond.

Finally, the rest of the lab takes the candidate out to lunch without me. This has a number of purposes. (1) The other lab members can figure out if the candidate is going to be able to be a good addition to the “team”. (2) The other lab members can tell the candidate the real truth about how our lab works and about my leadership style. This is important, because different trainees respond better or worse to different mentoring styles, and while I do have some dynamic range available to me, I do have my own style. I wouldn’t want someone to join my lab thinking they were going to be mentored with a style that I am incapable or unwilling to deliver. (3) The other lab members can tell me if they think the candidate is intellectually sound.

This method of combining an interview with the “manager”–me–and a less formal interview absent the manager with the other members of the team works really well.

Fatih Karakurt - February 2, 2010 at 6:58 am #

This is the best line I’ve read for a while:

…Like bacteria responding to antibiotics, strains of bad candidates that are immune to your process survive as well, and are hard to distinguish from good ones.

and explains how some people managed to get hired by some great firms.

Jill Elswick - February 19, 2010 at 6:57 am #

I remember a public speaker talking last year about employers not hiring the right people, simply because they didn’t like them. “There’s a person who could turn your company around, but you let him or her go,” he said. (I cannot remember the speaker’s name, unfortunately.)

But we want to work with people we like. It is very difficult to overcome that preference. Many people, I think, are also reluctant to hire those they think are smarter than they are. They’re okay with maintaining the quality status quo. Few people want to be shaken up in their thought or work processes. Also, if you make a bad hire of a super-intelligent person, your life will be hell. It’s a risk.

How do I work against the Winchester theory? I’ve stopped applying for jobs I don’t want. I evaluate potential bosses and co-workers on the basis of whether I would be inspired, motivated, and happy to work with them.

Dick - March 6, 2010 at 11:25 pm #

Over the years I’ve been the candidate a few times and been the interviewer at least a couple hundred times. I’ve made great hires, good hires and a couple awful ones (mostly engineers and various technical types). My thoughts about them paraphrase Churchill on democracy–terrible way to do business but the best humans have invented so far.

To protect a little against the weaknesses, I always do team interviews and include as wide a range of people as I can while keeping the group(s) reasonable. I’ve learned to listen very, very carefully to the feedback I get from the others and to pay attention to that “gut feel”.

And yes, there have been times when I knew 5 minutes into an interview we were or weren’t going to hire the person. Sometimes I consciously knew why and sometimes I didn’t–chemistry? fit? magic? got me but it was real.

It’s hard though because some folks interview well and some don’t and that’s not a very reliable predictor of success in the job.

Still, I think a well done interview can tell me a lot about the person’s energy and enthusiasm. It should also tell them a lot about whether the environment and the work will generate that energy and enthusiasm–where those line up, there’s a reasonable chance of success.