Frank L. Schmidt and John E. Hunter — we quote them a lot here at Plum. To many people, those names may not mean anything, but anyone remotely familiar with Industrial/Organizational Psychology knows Schmidt & Hunter have published some of the most influential research on valid talent selection methods to date.
To some, this may come as a shock — wait, there’s extensive research out there that proves how to identify and select top performing employees? Yes, there is — and resumes and cover letters have nothing to do with the selection methods proven by science. Schmidt & Hunter found that General Mental Ability (otherwise known as general intelligence or general cognitive ability) was the single best predictor of job performance. When combined with other valid selection tools, like personality/integrity tests and structured interviews, predictability only increases.
It's time for a talent revolution.
Reveal human potential to revolutionize how you hire, grow, and retain talent with Plum.
Although published over 20 years ago, their research is still extremely relevant today, which is why we’d love to take a moment to breakdown their findings for you. After all, as a talent professional, you’re a bit too busy to be nose-deep in long research papers. So we hope this 6 minute overview will help!
Before we dive in, let’s start with a primer to Industrial/Organizational Psychology — a research practice dedicated to making talent professionals’ lives easier (so why wouldn’t you want to learn more?).
What is Industrial/Organizational Psychology?
Industrial/Organizational (I/O) Psychology is the study of human behavior in the workplace. I/O Psychologists contribute to an organization’s success by improving performance, motivation, team effectiveness, job satisfaction, innovation, occupational health and well-being, and more. I/O Psychologists improve hiring, training, and management by studying worker behavior, evaluating companies, and conducting leadership training. I/O Psychology is one of the 15 recognized specialities in professional psychology in the United States.
Fortune 500 companies like Walmart, Amazon, and General Motors have in-house I/O Psychologists improving their employee selection, development, feedback, and more.
While the “organizational” side of I/O Psychology focuses on understanding how organizational structures and management styles affect individual behavior, the “industrial” side involves understanding how to best match individuals to specific jobs. A priority on this end of I/O Psychology is to gather evidence that identifies which selection methods best predict performance. Schmidt & Hunter’s research would fall into this category.
“The Validity and Utility of Selection Methods in Personnel Psychology,” explained
Frank L. Schmidt and John E. Hunter are American Industrial/Organizational Psychology PhDs, and in 1998 they published “The Validity and Utility of Selection Methods in Personnel Psychology.” What’s fascinating is that, although their research was published over 20 years ago, it remains one of the most influential publications on predictive talent selection methods to date (in fact, Schmidt, Oh, and Shaffer published an update to the 1998 publication in 2016 because of its influence in the I/O Psychology community). Even so, very few organizations have implemented their findings in their talent deployment processes.
Based upon their meta-analysis, Schmidt & Hunter argue three things:
-
The best predictors of performance are cognitive ability, work sample tests, personality tests, and structured interviews
-
Of these, cognitive ability is the #1 predictor of performance
-
Selection methods drastically impact business outcomes
Let’s dig into each of these 3 takeaways.
1. The best predictors of performance are cognitive ability, work sample tests, personality tests, and structured interviews
Schmidt & Hunter summarize the practical and theoretical implications of 85 years of research in personnel selection, studying the validity of 19 selection procedures for predicting job performance. They found that cognitive ability, work sample tests, personality/integrity tests, and structured interviews are the best predictors of performance. We’ll focus specifically on personality and cognitive ability, because that’s exactly what Plum measures.
When it comes to personality assessment, Schmidt & Hunter note that their definition of integrity correlates with factors of the Big 5 personality model (which includes conscientiousness, extraversion, agreeableness, neuroticism, and openness to experience. You can read more about the Big 5 model and how it relates to job performance here). Quote:
In terms of basic personality tests, integrity tests have been found to measure mostly conscientiousness, but also some components of agreeableness and emotional stability (Ones, 1993).
Although there have been other studies that demonstrate why different jobs require different personality types, the focus of Schmidt & Hunter is correlation to all types of jobs, rather than parsing the different correlations in different jobs (Side note: If you’re interested in I/O Psychology research that shows how different personalities predict performance in different types of jobs, we’d recommend Barrick and Mount’s meta-analysis).
Although personality was a very predictive indicator or performance, Schmidt & Hunter found that cognitive ability was the absolute best predictor of whether or not someone would succeed in a role.
2. Of these, cognitive ability is the #1 predictor of performance
Schmidt & Hunter found that General Mental Ability (GMA) has a very strong correlation to job performance, especially when compared to resumes. In behavioral science, correlations are measured using coefficients (r). These coefficients range from -1.0 to +1.0, and when it comes to selection tests, a coefficient of 0.35 and above is considered very beneficial.
Schmidt & Hunter found that that experience and education (what you’d find on a resume) only have a correlation of 0.18 and 0.10 with job performance respectively — correlation coefficients that are defined as “unlikely to be useful.” General Mental Ability, on the other hand, has a correlation coefficient of r=0.51 — meaning it’s predictive of on-the-job performance. Quote:
The research evidence for the validity of GMA measures for predicting job performance is stronger than that for any other method (Hunter, 1986; Hunter & Schmidt, 1996; Ree & Earles, 1992; Schmidt & Hunter, 1981). Literally thousands of studies have been conducted over the last nine decades. By contrast, only 89 validity studies of the structured interview have been conducted (McDaniel, Whetzel, Schmidt, & Mauer, 1994).
3. Selection methods drastically impact business outcomes
Schmidt & Hunter’s article also touches on the ROI of selection methods, and that the difference between success and failure in a business is based upon their selection methods. Schmidt & Hunter lay down some very quotable truth that we can’t help but quote verbatim (bolding our own):
In fact, many employers, both in the United States and throughout the world, are currently using suboptimal selection methods. For example, many organizations in France, Israel, and other countries hire new employees based on handwriting analyses by graphologists. And many organizations in the United States rely solely on unstructured interviews, when they could use more valid methods. In a competitive world, these organizations are unnecessarily creating a competitive disadvantage for themselves (Schmidt, 1993). By adopting more valid hiring procedures, they could turn this competitive disadvantage into a competitive advantage.
The validity of the personnel measure (or combination of measures) used in hiring is directly proportional to the practical value of the method—whether measured in dollar value of increased output or percentage of increase in output. In economic terms, the gains from increasing the validity of hiring methods can amount over time to literally millions of dollars. However, this can be viewed from the opposite point of view: By using selection methods with low validity, an organization can lose millions of dollars in reduced production.
Our purpose here is to make three important points: (a) the economic value of gains from unproven hiring methods are typically quite large, (b) these gains are directly proportional to the size of the increase in validity when moving from the old to the new selection methods, and (c) no other characteristic of a personnel measure is as important as predictive validity.
Conclusion
Schmidt & Hunter conclude their publication stating that “employers must make hiring decisions; they have no choice about that. But they can choose which methods to use in making those decisions.” They summarize the research evidence of their article examining combinations of methods that have different validities of predicting future job performance. Graphology has no validity. Education and experience have low validity. General Mental Ability and personality have high validity.
When it came to combining predictors of success, two stood out. The combination of a GMA test and a personality test had a composite validity of 0.65. The combination of a GMA test and a structured interview had a composite validity of 0.63.
Not only are these combinations highly valid, but they’re very practical, too. Both combinations can be used for entry-level and experienced applications, and both combinations are less expensive than many other combinations outlined in their research. “Hence, both are excellent choices,” Schmidt & Hunter state.
It’s a shame that Schmidt & Hunter’s research — which outlined in detail just how devastating the economic impacts of invalid selection methods (like resumes, and other indicators of experience and education) are — is disregarded by, or even unknown to, so many organizations. So many of us, as talent experts, are frustrated by the ROI and productivity costs of low performers and churn, while the solution to these frustrations has existed for over 20 years.
It’s because of Schmidt & Hunter’s (among other I/O Psychology research that reaches the same conclusions) insightful and influential research that we measure personality and two subcategories of GMA (fluid intelligence and social intelligence) here at Plum.
To learn more about how Plum brings valid and predictive selection methods backed by 50+ years of Industrial/Organizational Psychology research to organizations, book a demo with us!