Reporting - Item and Assessment Analysis

Item analysis report

The Item Analysis Report provides an in-depth Classical Test Theory (CTT) psychometric analysis of item performance. It enables you to drill-down into specific item statistics and performance data. The report includes key item statistics including item difficulty p-value, high-low discrimination, item-total correlation discrimination, item-rest correlation discrimination and item reliability. It also provides assessment statistics relating to the amount of time taken and the scores achieved.

The report includes an assessment level overview graph and summary table. The overview graph plots a single point for each item in the summary table. Each question is plotted in terms of its item difficulty p-value (X-axis) and by its item-total correlation discrimination (Y-axis).  The report also calculates item-rest correlation discrimination (the scatter-plot provides the option to plot "item-total" or "item-rest" along the Y-access):

Questionmark Item Analysis Report - Summary View

The summary table beneath the scatter plot graph contains a line item for every question on the assessment. The table provides information on the question order, question wording and description, and summary information regarding the item difficulty p-values and the item-total correlation discrimination for each question. You can select an item in the table to navigate to the details of the selected item. And you can sort on each column to get different views of question performance. For example, you can sort the questions by difficulty to look at the hardest questions at the top of the table.

This interactive report provides a high-level view of your items' performance and enables you to drill-down into specific item statistics and performance data. The report includes user-adjustable visual cues to indicate results that may need attention, graphical representation of how items compare with each other in terms of P-value and correlation, and several publish-to-PDF and publish-to-CSV options.

Item Analysis Report Item Analysis Report

View an example Item analysis summary report (summary of all item statistics in an assessment)

View an example Item analysis detail report (drills down to full item statistics for specific item)

Test analysis report

The Test analysis report presents detailed information regarding the statistical/psychometric performance, including the reliability, of an assessment. In short, the Test analysis report helps administrators ensure that a test is reliably measuring what is was designed to measure. Statistical analysis in the Test analysis report includes: Reliability (Cronbach’s Alpha); Skew; Kurtosis; Mean (arithmetic); Median; Mode; Standard Deviation; Variance; Standard Error of measurement; and Standard Error of the Mean. The report also presents the distribution of tests scores using a frequency distribution table and a histogram that graphically displays the range of scores. Click here for an overview of the Test analysis report (PDF).

  • Helps ensure that a test reliably measures what it was designed to measure.
  • Provides data and analysis crucial for ensuring defensibility of an assessment.
  • Calculates Cronbach’s Alpha Test Reliability - the gold standard for measuring reliability.
  • Distribution via browser, email, PDF, Word, Excel, and print.

Test Analysis Report

Assessment results reports

Assessment Results Over Time ReportAssessment results over time report (right) The Assessment results over time report provides information on the assessment results over a period of time specified by the user. It provides the mean, minimum and maximum values for an assessment, as well as 95% confidence interval of the assessment.

Survey frequency analysis report: The Survey frequency analysis report provides detailed frequency analysis information for each question on a survey assessment in terms of how many participants selected each answer. The report consists of a listing of each question on the survey and a graph showing the corresponding number of participants responding to each answer option.

Test center analysis report (right)
The Test center analysis report allows users to evaluate the performance of

participants that have been scheduled to take assessments at test centers so that users can review performance at various test center and evaluate whether there are potential test security issues at test centers.

analytics-assessment-completion-time.jpg

Assessment completion time report (right)
Valuable completion time analysis useful to:
  • Identify potential cheating
  • Flag potential content theft
  • Determine if allotted time is sufficient for completing the assessment

​Demographic report: Review a breakdown of assessment results by a demographic variable

Question frequency analysis report: Get quick basic view of response frequency for all questions in a given assessment

Item bank reports

Question type report: (right) This report is an item bank report that provides information on how many questions you have in your repository by question type (e.g., Multiple Choice, Yes/No, etc.). You can see the information for your entire repository or just for specific topics that you specify. The CSV version of this report gives detailed information about the questions wording, description, type, and the location of questions in topics.

Question status report: This report is an item bank report that provides information on how many questions you have in your repository by question status. You can see the information for your entire repository or just for specific topics that you specify. The CSV output options provide detailed information about the questions wording, description, their status,and location in topics.

Item bank reports

Analytics report filtering and output options

The Analytics tab in the user interface provides direct access to several reports with a variety of filtering (e.g. by group, date range, demographics) and distribution options (e.g., browser, CSV, PDF).

Questionmark Analytics: Select assessment to report on; set filters; choose distribution method