Creating Valid, Reliable and Trustable Assessments | Questionmark

Creating Valid, Reliable and Trustable Assessments

Two Day Course

All tests should be well developed and testing practices beneficial. There is extensive evidence documenting the effectiveness of well-constructed tests in relation to supporting the validity of the test. Improper use of tests or interpretation of scores, however, can cause considerable harm to test takers as well as others affected by test-based decisions. This powerful two-day course provides instruction and engages participants in the design and development of test items and assessments. The course includes the following topics:

  • The Test Development Process
  • Development of Learning Objectives
  • Topic Structure Development
  • Importance and Development of a Test Plan
  • Cognitive Levels of Testing and Item Complexity
  • Use of Correct Grammar and Effective Language
  • Test Item Development
  • Determining Cut/Passing Scores
  • Reporting and Analytics
  • Developing Pre- and Posttests

The active learning process will be used to ensure participants are fully engaged. Participants will be given the opportunity to develop test items using the Questionmark Assessment Management System. This course is designed for anyone involved in the test development, review and administration process, from novice to seasoned professionals.

Course Dates and Registration

Registration details:October 16 & 17, 2017
Questionmark Training Center, Parkside Tower, Suite 825, 215 South State Street, Salt Lake City, UT

Register now
Tuition:$995 (includes tuition and course materials)
Hours:9:00 am to 4:00 pm each day
Breaks:15 minute break in morning; 1 hour for lunch; 15 minute in afternoon

Course Outline

The following high-level learning objectives associated with the course content will be covered:

Testing Process

  • Describe the testing process
  • Explain the roles of participants in the testing process

Test Development Process

  • Describe the purpose of testing
  • List the qualities of a successful test
  • List the two general forms of tests
  • List the three levels of criticality of tests
  • Discuss the elements of successful test construction

Development of Objectives

  • List the three elements of a terminal/final performance objective
  • Use Bloom’s “new” taxonomy to develop objectives

Topic Structure Development

  • Develop a topic structure within the Questionmark Assessment Management System

Development of a Test Plan

  • Develop a test plan

Cognitive Levels of Testing and Item Complexity

  • Describe the cognitive levels of learning and testing
  • Describe levels of item complexity associated with various levels of cognition

Use of Correct Grammar and Effective Language

  • Use correct grammar to develop meaningful test items
  • Use effective language to develop meaningful test items

Test Item Development

  • Develop test items using three methods of test item development
  • Describe the qualities associated with each part of a test item
  • Describe the qualities of a “good” test item
  • Write test items

Performance Testing

  • Describe performance testing

Determining Cut/Passing Scores

  • Describe the importance of setting cut scores
  • Describe widely accepted methods for setting an assessment cut score

Reporting and Analytics

  • Interpret test score distributions
  • Interpret commonly reported test statistics

Developing Pre- and Posttests

  • Describe the purpose of pre- and posttests

About the Instructor

James R. Parry, M.Ed. CPT
Owner and Chief Executive Manager, Compass Consultants

Jim has over 38 years experience as a course designer, developer and instructor. Jim has partnered with Questionmark over the years both as a subject matter expert and previous customer. He has presented numerous assessment best practice trainings and at Questionmark User's Conference.

About the Instructor