Authoring Best Practice General Psychometrics Standards

Know what your questions are about before you deliver the test

Posted by Austin Fossey A few months ago, I had an interesting conversation with an assessment manager at an educational institution—not a Questionmark customer, mind you. Finding nothing else in common, we eventually began discussing assessment design. At this institution (which will remain anonymous), he admitted that they are often pressed... Read More

Authoring Best Practice General Psychometrics Standards

Simpson’s Paradox and the Steelyard Graph

Posted by Austin Fossey If you work with assessment statistics or just about any branch of social science, you may be familiar with Simpson’s paradox—the idea that data trends between subgroups change or disappear when the subgroups are aggregated. There are hundreds of examples of Simpson’s paradox (and I encourage you... Read More

Authoring Best Practice Innovative Technologies Psychometrics Standards

Is There Value in Reporting Subscores?

Posted by Austin Fossey The decision to report subscores (reported as Topic Scores in Questionmark’s software) can be a difficult one, and test developers often need to respond to demands from stakeholders who want to bleed as much information out of an instrument as they can. High-stakes test development is lengthy... Read More

Best Practice General Psychometrics Standards

Item Development – Summary and Conclusions

Posted by Austin Fossey This post concludes my series on item development in large-scale assessment. I’ve discussed some key processes in developing items, including drafting items, reviewing items, editing items, and conducting an item analysis. The goal of this process is to fine-tune a set of items so that test developers... Read More