How do you know what time limit to set for a test or exam? I’ve presented a webinar on some tips on how you can improve your tests and exams (you can view the recording here, 10 Quick Tips to Improve Your Tests and Exams) and this is one of the subjects covered. This blog also gives some good practice on setting a time limit.
Power tests
The first thing to identify is what the test is seeking to measure, and whether this has a speed element. Most tests are “power” tests in that they seek to measure someone’s knowledge or skill, not how fast it can be demonstrated. In a power test, you could set no time limit, but for practical purposes, it’s usual to set a time limit. This should allow most people to have enough time to complete answering the questions.
The best way to set a time limit is to pilot the test and measure how long pilot participants take to answer questions and use this to set an appropriate time period. If you have an established testing program, you may have organizational guidelines on time limits, for example you might allow a certain number of seconds or minutes per question; but even if you have such guidelines, you must still check that they are reasonable for each test.
Speed tests
Sometimes, speed is an important part of what you are trying to measure, and you need to measure that someone not only can demonstrate knowledge or skill but can also do so quickly. In a speed test, failure to be able to answer quickly may mean that the participant does not meet the requirements for what is being measured.
For example, in a compliance test for bank personnel to check their knowledge of anti-bribery and corruption laws, speed is probably not part of what is being measured. It will be rare in practice for people to encounter real-life issues involving bribery and very reasonable for them to think and consider before answering. But if you are testing a medical professional’s ability to react to a critical symptom in a trauma patient and make a decision on a possible intervention, rapid response is likely part of the requirement.
When speed is part of the requirements of what is being measured, the time limit for the test should be influenced by the performance requirements of the job or skill being measured.
Monitoring time limits
For all tests, it is important to review the actual time taken by participants to ensure that the time limit remains appropriate. You should regularly check the proportion of participants who answer all the questions in the test and those who skip or miss out some questions. In a speed test, it is likely that many participants will not finish the test. But if many participants are failing to complete a power test, then this should be investigated and may mean that the time limit is too short and needs extending.
If the time limit for a power test is too short, then essentially it becomes a speed test and is measuring how fast participants can demonstrate their skills. As such, if this is not part of the purpose of the test, it will impact the validity of the test results and it’s likely that the test will mis-classify people and so be unfair.
A particular point of concern is when you are using computerized tests to test people who are not proficient computer users. They will inevitably be slower than proficient computer users, and unless your test seeks to measure computer proficiency, you need to allow such people enough time.
What about people who need extra time?
It’s common to give extra time as accommodation for certain kinds of special needs. Extra time is also sometimes given for linguistic reasons e.g. taking an assessment in second language. Make sure that your assessment system lets you override the time limit in such cases. Ideally base the extra time in such cases on piloting, not just a fixed extra percentage.
When should a time limit start?
My last tip is that the time limit should only start when the questions begin. If you are presenting any of these:
- Introductory material or explanation
- Practice questions
- An honor code to commit to staying honest and not cheating
- Demographic questions
The time limit should start after these are done. If you are using Questionmark software, you can make this happen by excluding the question block from the assessment time limit.
If you are interested in more tips on improving your tests and exams, you can view the webinar here: 10 Quick Tips to Improve your Tests and Exams.