Conference 2019 Session: Increasing Defensibility of High-stakes Tests Through Job Analysis

Increasing Defensibility of High-stakes Tests Through Job Analysis

Content Track:  Assessments in Practice    
Jone Papinchock, Ph.D., Senior Personnel Psychologist | Federal Deposit Insurance Corporation (FDIC)

Thorough job analysis is often an overlooked step in the test development process. This is particularly problematic in high-stakes employment testing where test content must be directly tied to competencies that are important and needed at entry to the job. I will present information on what a job analysis is, the authorities that describe defensible job analysis practices in the U.S. (Uniform Guidelines for Employee Selection Procedures, APA Standards on Educational and Psychological Testing, and SIOP Principles for the Validation and Use of Selection Procedures), how to create an effective job analysis questionnaire, how to design research to collect reliable job analysis data, parameters to use in analysing the data and creating test plans, and the litigation ramifications of failing to conduct and document the job analysis.

Throughout I will provide examples from my experience as a consulting/testifying expert in class action employment discrimination litigation including job analysis and content validity issues and as a testing consultant in private and public sector test development projects. Practical considerations about setting job analysis criteria; the impacts of response rate, sample size, and standard deviations; and how these elements are scrutinized by the Courts will be discussed. 

Learning Objectives 
  • Establish the critical components of conducting a job analysis that will withstand legal scrutiny.

  • Define the authorities that govern legally defensible job analysis and content validity strategies in test development.

  • Describe practical steps for collecting and interpreting reliable job analysis data.

  • Provide tools for linking job content to test content.

  • Demonstrate the impact of job analysis quality on litigation outcomes.