I’ve spent a lot of time in the last six months reading studies. On recruiting. And interviewing. And diversity. And Quality of Hire. And I’ve mixed it with a healthy dose of Nassim Taleb, LessWrong.com and the work of people like Daniel Pink, Robert Cialdini, and Daniel Kahneman. 

What I’ve concluded is that “Business” is not that complex. Human beings are complex. And that’s where everything falls apart.  

Academic Studies

Let’s talk about academia, and how it impacts recruiting. In 2012, Lauren Rivera published a study of elite professional consulting firms and their hiring practices, The study purported to show that when hiring, the choices made by a manager ignored objective decision making and ultimately revolved around social ties. A popular summary is that squash players hired squash players and golf players hired golf players. 

Now that’s a headline. But is that what the data showed? Is that even what Rivera claimed, or is that how it was reported? 

One of the challenges in social science is the desire to draw conclusions from data. Data is simply information, and it is accurate or inaccurate. Conclusions taken from data are trickier. They can be right, but wrong. They can correctly predict an outcome, but not be accurate. And it’s easy to show. If I sit in a certain chair every time the Cowboys play football, they win. If I don’t sit in that chair, they lose. I can show data that proves this is true, and it can be 100% predictive. It’s clearly not the case that my choice of sitting in a chair can predict the outcome of a football game I’m not involved in, but you can draw an accurate conclusion based on observation if it works. The conclusion is correct in its prediction but wrong in its accuracy. We were “Fooled by Randomness“. 

It’s likely accurate to say that “objective processes” at the kinds of companies Rivera studied weren’t objective. The hiring process at the companies were not data-driven. We can conclude this because that was the point of the study. It’s not accurate to then move on and conclude that squash players hired squash players and golf players hired golf players as part of the crucial factor in decision making. That’s the sexy part of the story, but they didn’t test for that conclusion**(_see below_). 

The “they play golf” marker is not a rational decision on how to hire for high-value positions. That’s why it stands out in the data. As a correlation, Rivera actually concludes that managers seek to find a “spark of commonality” that gave the managers assurance that the candidate would be a good culture fit. If we’re having a discussion and I talk about a book I read, a movie I watched, a restaurant I visited, and a company I worked for, you’re likely to engage on the topic that you know the most about. If we both read the same book, we can talk about the book. Someone observing us would assume the book was the central part of the discussion, but it’s actually the one topic we settled on out of four topics.   

And here’s the kicker – whether its squash or Twilight, a shared interest is a sign of cultural fit. 

When we look at high-performing teams, a common pattern is similar communication patterns. The team has a high degree of confidence in their ability to understand what the other team members are saying. A shared language, a shared vocabulary, and a shared sense of motivation allow the team to solve problems without having to spend time explaining themselves or managing internal conflict.  When hiring, managers who search for common interests and a common language, they are screening for someone who will be a good cultural fit. 

What about Diversity? 

Studies clearly indicate that diversity in a team is good for identifying weakness in an argument or a clear bias. If everyone on a team thinks the same way, they are likely to blindly make poor decisions. What’s left out of the reporting on this topic is that the more diversity you have, the more conflict you will encounter. A diverse team is a more self-aware team, but a less efficient team. 

Like all things in life – there is a balance. You want a consistency of cultural fit that creates a smooth running team, with a diversity of background and perspective that allows for a broader view of the problem.

What is the manager’s job?

To hire and manage a smooth running team. What is the best way to do that? Hire people with a commonality of thought, or hire people who can learn to work together. Managers searching for a cultural fit have a rational reason to do so, even if they ultimately appear to rely on shortcuts like what sports you play. We may try to create an objective process that hires the best person for the job, but the rational process of the manager is a desire to hire teams that communicate effortlessly.

What A Study Can Tell You And What It Can’t

Rivera’s work showed that the “objective” hiring processes put into place by the firms she observed and collected data from, were not primarily responsible for the decision to hire. 

That’s it. That is the full extent of the data. That’s all it can tell us. 

1) The study cannot definitively tell us what other firms did. 

2) The study cannot definitively tell us the primary reasons used by a manager for hiring. 

3) The study cannot definitively tell us the best methods for hiring. 

4) The study cannot definitively tell us this hiring process was less effective than another process. 

But man, we really want it to, don’t we? A study, conducted correctly, is really only good at proving something is not true. Once we get to conclusions, we’re injecting our bias into the results.

Don’t take my word for it. Consider funding a study to see if I’m wrong. 

**_To test the squash theory, you would have to run a series of interviews where squash and non-squash players were in four control groups. A series of interviews where the subject of squash came up or did not come up would then be compared to the control groups where it was brought up. A clear result might show that there was no difference. That would eliminate the squash reference. If the manager hired squash players no matter whether it was brought up or not, it would show there was a hidden reason we failed to identify. If the only group of the four that was hired were the people who played squash in an interview in which it was brought up, that would be at best be an example of ingroup bias. In terms of hiring, this ingroup bias of playing sports could be a better indicator of cultural fit than any other selection method we make up. 

General News Recruiting
Authors
Jim Durbin

Jim Durbin is managing principal at Texas-based Brandstorming, which describes itself as ":equal parts full-service digital agency, training company and recruiting firm." He's known as a Social Media Headhunter, because he places high level marketing managers and because he's trained over 9,000 recruiters and salespeople on digital tools. Follow with him on Twitter or connect with him on LinkedIn.