Psychometrics

Posted by John Kleeman I’m running a session at the Questionmark user conference next month on Item Analysis for Beginners and thought I’d share the answer to an interesting question in this blog. When you run an Item Analysis report, one of the useful statistics you get on a question... Read More

Posted by Julie Delazyn Wendy Lau is a Psychometrician at Pacific Gas and Electric Company (PG&E). She will be leading a discussion at Questionmark Conference 2016 in Miami, about Safety and the Utilities Industry: Why Assessments Matter. [caption id="attachment_16422" align="alignright" width="244"] Wendy Lau, Psychometrician, PG&E[/caption] Wendy’s session will describe a... Read More

Posted by Austin Fossey There are many fantastic books about test development, and there are many standards systems for test development, such as The Standards for Educational and Psychological Testing. There are also principled frameworks for test development and design, such as evidence-centered design (ECD). But it seems that the... Read More

Posted by Austin Fossey One of our webinar attendees recently emailed me to ask if there is a way to calculate reliability when items are randomly selected for delivery in a classical test theory (CTT) model. As with so many things, the answer comes from Lee Cronbach—but it’s not Cronbach’s Alpha.... Read More

Posted by Austin Fossey I had the privilege of meeting with an organization that is reporting subscores to show how their employees are improving across multiple areas of their domain, as determined by an assessment given before and after training. They have developed some slick reports to show these scores, including... Read More

Posted by Austin Fossey In my last post, I discussed five ways you can limit the use of breached content so that a person with unauthorized access to your test content will have limited opportunities to put that information to use; however, those measures only control the problem of a content... Read More

Posted by Austin Fossey In an earlier post, Questionmark’s Julie Delazyn listed 11 tips to help prevent cheating. The third item on that list related to minimizing item exposure; i.e., limiting how and when people can see an item so that content will not be leaked and used for dishonest purposes.... Read More

Posted by Austin Fossey In my last post, I talked about how item discrimination is the primary statistic used for item selection in classical test theory (CTT). In this post, I will share an example from my item analysis webinar. The assessment below is fake, so there’s no need to write... Read More

Regular readers of our blog know that we ran an initial series on item analysis way back in the day, and then I did a second item analysis series building on that a couple of years ago, and then I discussed item analysis in our item development series, and then... Read More

Posted by Austin Fossey A few months ago, I had an interesting conversation with an assessment manager at an educational institution—not a Questionmark customer, mind you. Finding nothing else in common, we eventually began discussing assessment design. At this institution (which will remain anonymous), he admitted that they are often pressed... Read More