Here’s some basic information about the Item Analysis Report, which was recently added to Questionmark Analytics:
What it does: The item analysis report provides an in-depth Classical Test Theory psychometric analysis of item performance. It enables you to drill-down into specific item statistics and performance data. The report includes key item statistics including item difficulty p-value, high-low discrimination, item-total correlation discrimination and item reliability. It also provides assessment statistics relating to the amount of time taken and the scores achieved.
Who should use it: Assessment, learning and education professionals can use this report to determine how well questions perform psychometrically.
How it looks: The report includes an assessment level overview graph and summary table. The overview graph plots a single point for each item in the summary table. Each question is plotted in terms of its item difficulty p-value (X-axis) and by its item-total correlation discrimination (Y-axis):
- Questions that have high (acceptable) discrimination will appear near the top of the graph
- Questions with low (unacceptable) discrimination will appear at the bottom of the graph
- Difficult questions will appear to the left of the graph
- Easy questions will appear to the right of the graph
The summary table beneath the scatter plot graph contains a line item for every question on the assessment. The table provides information on the question order, question wording and description, and summary information regarding the item difficulty p-values and the item-total correlation discrimination for each question. You can select an item in the table to navigate to the details of the selected item. And you can sort on each column to get different views of question performance. For example, you can sort the questions by difficulty to look at the hardest questions at the top of the table.
By clicking on any of the rows in the summary table one can go to a detailed item view of question-level information.