Psychometrics Archives | Questionmark

What is predictive validity?

Posted on Juni 30, 2021 (Juli 13, 2021) by John Kleeman under Articles

Posted by John Kleeman, Founder Predictive validity is how well a test score can predict scores in other metrics. The concept features in psychometrics and is used in a range of disciplines such as recruitment. Glossary of terms This mini glossary will explain certain terms used throughout the article. Validity... Read More

Psychometrics

Item Analysis for Beginners – When are very Easy or very Difficult Questions Useful?

Posted on Februar 20, 2018 (Januar 20, 2022) by John Kleeman under Articles

Posted by John Kleeman I’m running a session at the Questionmark user conference next month on Item Analysis for Beginners and thought I’d share the answer to an interesting question in this blog. When you run an Item Analysis report, one of the useful statistics you get on a question... Read More

Events General Psychometrics Q&A's Security

The Importance of Safety in the Utilities Industry: A Q&A with PG&E

Posted on März 16, 2016 by John Kleeman under Articles

Posted by Julie Delazyn Wendy Lau is a Psychometrician at Pacific Gas and Electric Company (PG&E). She will be leading a discussion at Questionmark Conference 2016 in Miami, about Safety and the Utilities Industry: Why Assessments Matter. [caption id="attachment_16422" align="alignright" width="244"] Wendy Lau, Psychometrician, PG&E[/caption] Wendy’s session will describe a... Read More

Authoring Best Practice Psychometrics Standards

Checklists for Test Development

Posted on Dezember 15, 2015 by John Kleeman under Articles

Posted by Austin Fossey There are many fantastic books about test development, and there are many standards systems for test development, such as The Standards for Educational and Psychological Testing. There are also principled frameworks for test development and design, such as evidence-centered design (ECD). But it seems that the... Read More

Authoring Best Practice General Interoperability Psychometrics

G Theory and Reliability for Assessments with Randomly Selected Items

Posted on November 17, 2015 (Juli 19, 2021) by John Kleeman under Articles

Posted by Austin Fossey One of our webinar attendees recently emailed me to ask if there is a way to calculate reliability when items are randomly selected for delivery in a classical test theory (CTT) model.As with so many things, the answer comes from Lee Cronbach—but it’s not Cronbach’s Alpha. In... Read More

Best Practice Psychometrics Standards

Is There Value in Reporting Changes in Subscores?

Posted on November 11, 2015 by John Kleeman under Articles

Posted by Austin Fossey I had the privilege of meeting with an organization that is reporting subscores to show how their employees are improving across multiple areas of their domain, as determined by an assessment given before and after training. They have developed some slick reports to show these scores, including... Read More

Authoring Best Practice Events General Psychometrics Standards

4 Ways to Identify a Content Breach

Posted on Oktober 27, 2015 by John Kleeman under Articles

Posted by Austin Fossey In my last post, I discussed five ways you can limit the use of breached content so that a person with unauthorized access to your test content will have limited opportunities to put that information to use; however, those measures only control the problem of a content... Read More

Best Practice Psychometrics Standards

5 Ways to Limit the Use of Breached Assessment Content

Posted on Oktober 20, 2015 by John Kleeman under Articles

Posted by Austin Fossey In an earlier post, Questionmark’s Julie Delazyn listed 11 tips to help prevent cheating. The third item on that list related to minimizing item exposure; i.e., limiting how and when people can see an item so that content will not be leaked and used for dishonest purposes.... Read More

Authoring Best Practice Events General Psychometrics

Item analysis: Selecting items for the test form – Part 2

Posted on September 29, 2015 by John Kleeman under Articles

Posted by Austin Fossey In my last post, I talked about how item discrimination is the primary statistic used for item selection in classical test theory (CTT). In this post, I will share an example from my item analysis webinar. The assessment below is fake, so there’s no need to write... Read More

Best Practice Events General Psychometrics Standards

Item analysis: Selecting items for the test form – Part 1

Posted on September 23, 2015 by John Kleeman under Articles

Regular readers of our blog know that we ran an initial series on item analysis way back in the day, and then I did a second item analysis series building on that a couple of years ago, and then I discussed item analysis in our item development series, and then... Read More