Authoring General Psychometrics Standards

Discussing the revised Standards for Educational and Psychological Testing

Posted by Austin Fossey I just returned from the National Council for Measurement in Education (NCME) annual meeting in Philadelphia, which is held in conjunction with the American Educational Research Association (AERA) annual meeting. There were many big themes around research and advances in assessment, but there were also a lot... Read More

Authoring Best Practice General Interoperability Psychometrics Standards

Item Analysis Report – Item Reliability

Posted by Austin Fossey In this series of posts, we have been discussing the statistics that are reported on the Item Analysis Report, including the difficulty index, correlational discrimination, and high-low discrimination. The final statistic reported on the Item Analysis Report is the item reliability. Item reliability is simply the product... Read More

Assessments in Practice Best Practice General Psychometrics

Item Analysis – Two Methods for Detecting DIF

Posted by Austin Fossey My last post introduced the concept of differential item functioning. Today, I would like to introduce two common methods for detecting DIF in a classical test theory framework: the Mantel-Haenszel method and the logistic regression method. I will not go into the details of these two methods,... Read More

Best Practice General Psychometrics

Item Analysis Report – High-Low Discrimination

Posted by Austin Fossey In our discussion about correlational item discrimination, I mentioned that there are several other ways to quantify discrimination. One of the simplest ways to calculate discrimination is the High-Low Discrimination index, which is included on the item detail views in Questionmark’s Item Analysis Report. To calculate the... Read More

Best Practice Events General Psychometrics

Psychometrics and Measurement Design: A conversation

Posted by Joan Phaup Many delegates to the Questionmark 2014 Users Conference in San Antonio March 4 – 7 want to learn about assessment-related best practices. Austin Fossey, our Reporting and Analytics Manager, will talk about Principles of Psychometrics and Measurement Design during one of the many breakout sessions on... Read More

Best Practice Events General Psychometrics Q&A's Writing Assessments

Item Analysis Report – Item Difficulty Index

Posted by Austin Fossey In classical test theory, a common item statistic is the item’s difficulty index, or “p value.” Given many psychometricians’ notoriously poor spelling, might this be due to thinking that “difficulty” starts with p? Actually, the p stands for the proportion of participants who got the item... Read More

Authoring Best Practice Interoperability Psychometrics Standards

Conceptual Assessment Framework: Building the Evidence Model

Posted by Austin Fossey In my previous posts, I introduced the student model and the task model—two of the three sections of the Conceptual Assessment Framework (CAF) in Evidence-Centered Design (ECD). The student and task models are linked by the evidence model. The evidence model has two components: the evaluation... Read More