What is Predictive Validity?

Predictive validity is how well a test score can predict scores in other metrics. The concept is featured in psychometrics and is used in a range of disciplines such as recruitment.

Glossary of terms

This mini glossary will explain certain terms used throughout the article.

• Validity – How accurate a concept measures a desired criterion in a test or study
• Test score – A measure that demonstrates the performance of someone who takes a test, often given as a number or grade
• Scale – A measure or order of things in relation to specified criteria. For example, a rating scale of customer satisfaction, measured on a scale of 1-10 where 1 is the least satisfied and 10 is the most satisfied.
• Utility – Also known as “practical value”, utility is a measure of worth or value in a study or test.
• Cognitive test – An assessment of a candidate’s cognitive abilities.
• Psychometrics – The study of psychological measurement including its theory and techniques and the measurement of criteria such as knowledge, attitudes, and personality.

How does it work?

In recruitment, predictive validity examines how appropriately a test can predict criteria such as future job performance or candidate fit. A way to do this would be with a scatter plot.

In this scatter plot diagram, we have cognitive test scores on the X-axis and job performance on the Y-axis. The horizontal line would denote an ideal score for job performance and anyone on or above the line would be considered “successful”.

These diagrams can tell us the following:

• The difficulty of the test.
• The correlative relationship between test scores and a desired measure (job performance in this example). A weak positive correlation would suggest low validity and a difficulty in finding optimal candidates while a strong positive correlation would suggest high validity and an easier way to find the right candidates.
• Where the ideal score line should be placed.

Other examples of predictive validity

• IQ tests that predict the likelihood of candidates obtaining university degrees several years in the future.
• Personality tests that predict future job performance.
• Depression outcome tests that predict potential behaviors in people suffering from mental health conditions.
• Maths exams that predict success in the sciences.

Advantages and disadvantages of predictive validity

Advantages

• Predictive validity has been shown to demonstrate positive relationships between test scores and selected criteria such as job performance and future success.
• Successful predictive validity can improve workforces and work environments. For example, a study examining the predictive validity of a return-to-work self-efficacy scale for the outcomes of workers with musculoskeletal disorders showed that periodic telephone interviews helped to determine when they were ready to return. Theoretically, this could also lead to better accommodations for disabled workers, improved morale, and greater care for all employees.

Disadvantages

• It can take a while to obtain results, depending on the number of test candidates and the time it takes to complete the test.
• Biases and reliability in chosen criteria can affect the quality of predictive validity.

Concurrent validity vs predictive validity

There are multiple forms of statistical and psychometric validity with many falling under main categories. One other example is concurrent validity, which, alongside predictive validity, is grouped by criterion validity as they use specific criteria as part of their analyses.

Concurrent validity examines how measures of the same type from different tests correlate with each other. The main difference between concurrent validity and predictive validity is the former focuses more on correlativity while the latter focuses on predictivity.

Other forms of validity:

• Criterion validity checks the correlation between different test results measuring the same concept (as mentioned above).
• Construct validity checks how effectively a test measures the idea that it’s made to measure.
• Content validity checks the test content to see how well it represents a sample of its measured behavior(s).
  • Face validity is a form of content validity and checks the suitability of the test content in relation to its goals
• The following are classed as “experimental”:
  • Statistical conclusion validity checks the correctness of conclusions made about the relationship between test variables.
  • Internal validity checks the approximate conclusions made about cause and effect, based on test measurements, research, and methodology.
  • External validity checks how test results can be used to analyze different people at different times outside the completed test environment.

Biases in predictive validity

Biases can play a varying role in test results and it’s important to remove them as early as possible. By doing this, you ensure accurate results that keep candidates safe from discrimination.

An example of a bias is basing a recruitment decision on someone’s name, appearance, gender, disability, faith, or former employment. These biases can take place before or during an interview or test process and can significantly affect predictive validity. This is why personality tests aren’t always efficient for all cases.

Another example of bias could be the perception that higher levels of experience correlate with innovation. This does not always match up as new and positive ideas can arise anywhere and a lack of experience could be the result of factors unrelated to one’s ability or ideology.

Combating biases can be difficult but it’s an important step for the safety of test candidates and employees as well as the efficiency of a business and its workforce.

Conclusion

It’s an ongoing challenge for employers to make the best choices during the recruitment process. As recruiters can never know how candidates will perform in their roles, measures like predictive validity can help them choose appropriately and enhance their workforce.

Discover how Questionmark’s online assessment tools can help with that by providing secure, reliable, and accurate assessment platforms and results. Talk to us today.