At the Questionmark Users Conference, I recently spoke about a subject dear to my heart – psychometrics. Here’s a sneak preview on one of my topics: item total correlation! What is it, and what does it mean?
What is the item total correlation, and what does it mean?
The item total correlation is a correlation between the question score (e.g., 0 or 1 for multiple choice) and the overall assessment score (e.g., 67%).
It is expected that if a participant gets a question correct they should, in general, have higher overall assessment scores than participants who get a question wrong.
Similarly, with essay type question scoring – where a question could be scored between 0-5 participants who did a really good job on the essay (got a 4 or 5) should have higher overall assessment scores (maybe 85-90%). This relationship is shown in an example graph below:
This relationship in psychometrics is called ‘discrimination’; referring to how well a question differentiates between participants who know the material and those that do not know the material.
How to test item total correlation
Participants who know the material taught to them should get high scores on questions and high overall assessment scores. Participants who did not master the material should get low scores on questions and lower overall assessment scores.
This is the relationship that an item total correlation provides to help evaluate the performance of questions. We want to have lots of highly discriminating questions on our tests because they are the most fine-tuned measurements to find out what participants know and can do.
This can be critical to unlocking the full value of online assessments.
What to look out for when testing item total correlation
When looking at an item-total correlation, generally negative values are a major red flag. For example, participants who get low scores on the questions would not be expected to get high scores on the assessment. This could indicate a mis-keyed question or that the question was highly ambiguous and confusing to participants.
Values for an item-total correlation (point-biserial) can also help indicate discrimination in your questions:
- values between 0 and 0.19 may indicate that the question is not discriminating well
- values between 0.2 and 0.39 indicate good discrimination
- values 0.4 and above indicate very good discrimination
Get more out of your assessments
Questionmark offers an enterprise-grade assessment platform delivered with care and unequalled expertise—delivered in the cloud or on premise.
We also offer professional assessment authoring, audits of existing assessments and much more to help firms ensure the quality of their assessments.
Get in touch to discuss your needs, or request a free demonstration today .