Many organizations need to deliver assessments in the workplace for an array of reasons, including meeting compliance requirements, identifying skill gaps, providing certifications and much more. Most organizations use an LMS (Learning Management System) to do this, and it’s an excellent tool for anyone managing L&D within their company. The common question, however, is whether to use only an LMS to deliver quizzes, tests and exams or else to use it in conjunction with an Assessment Management System (AMS).
Get in touch
Talk to us about how we can elevate your LMS experience.
An LMS usually only has basic integral assessment functionalities. In order to enhance your LMS for better capability to create, deliver and report on assessments, you can integrate with a AMS.
This article explains the pros and cons of using an LMS for assessments or using an LMS in conjunction with an AMS.
Quick Links
- Why organizations choose an LMS for their assessments
- Why organizations choose an LMS + AMS for their assessments
- Reliability, validity, fairness and security
- Pros of using just an LMS for assessments
- Pros of using an LMS + AMS for assessments
- Summary
Why organizations choose an LMS for their assessments
Organizations tend to choose to deliver assessments from their LMS alone if several of the following factors apply:
- They only need tests and assessments for learning purposes (they aren’t used for making people decisions)
- The tests needed are low-stakes
- Only simple question types are used or needed (often multiple choice)
- There’s little concern around test cheating
- The industry in question is lightly regulated
- They don’t need tests to be delivered in multiple languages
- The LMS in question has a higher level of assessment functionality than is typical
Why organizations choose an LMS + AMS for their assessments
Organizations tend to choose to deliver assessments from an AMS in conjunction with an LMS if several of the following factors apply:
- Their test results matter (are used for important decisions about people)
- There’s concern about test cheating
- Tests are medium or higher-stakes
- The industry in question is heavily regulated
- There’s a focus in making tests more fair, inclusive, and diverse
- They see the value of re-using questions in multiple tests
- They have professional learning and assessment experts working for them
- They’re concerned with measuring skills accurately and effectively
- They need specific capabilities found in an AMS (e.g. translation tools, proctoring options, observational assessments, etc.)
Reliability, validity, fairness and security
Before listing the pros and cons, it’s worth considering what is needed to make well-informed business decisions from assessment results. There are four key factors: reliability, validity, fairness and security.
- Reliability means whether the assessment is consistent, i.e. it measures the same thing consistently and reproducibly. In order for a test or exam to be reliable, there needs to be, amongst other things, a predictable authoring process, trustworthy delivery and effective analysis.
- Validity means whether the test scores can be validly used to determine what the test seeks to measure. For example, if a test measures competence at a workplace skill, does it do so fully and properly? For instance, if someone passes the test, then are they considered competent?
- Fairness means that the assessment is applicable to all different categories and demographics of test takers. A fair test will measure only the things it seeks to measure and will not be biased against some of the people taking it.
- Security means that people should take the test without fraud or cheating, so that the test genuinely measures their capability. If there is widespread cheating, test results are not that meaningful.
Of course, it’s not just about the tools. To get a job done well, you need good tools, but you also need to design and write your assessments well. A good tool will help do this, but it needs effective use.
But let’s move onto the pros and cons. Each organization will have different circumstances and LMSs vary widely in how much in-built assessment capability they have. So not all the pros and cons may be applicable to every organization and the weighting each organization puts on the factors will vary considerably. But we hope this article will help illuminate the choice.
Pros of using just an LMS for assessments
Here are the main advantages of using an LMS’s in-built assessment tools:
1. Reduced cost
Since your organization has already purchased the LMS, and typically there is no extra charge to use its assessment capabilities, using the LMS for assessment is “free”.
2. Less administrative overhead
For every IT system you maintain, there are purchasing, security and legal issues to procure and maintain it, plus you need to configure and manage the system and update the configuration as the system changes. If you only have one system, an LMS, not two, an LMS and an AMS, there is less of this to do.
3. No need for integration
If you use both an LMS and AMS, there will be technical integration between the two. Although this is usually easy to set up, it is an extra step to consider.
4. Less friction for authors and administrators
Using only an LMS, an author can create an assessment and configure it directly within the LMS. If using an AMS with an LMS, the assessment author uses the AMS to create the assessment and then sets it up in the LMS.
5. Less training required
Authors and administrators only need training on how to use the LMS, not on the AMS in addition.
6. Cleaner interface for learners
In some LMSs, it may be easier to embed assessments within learning if native assessments are used. The LMS may also be able to better send reminders and notifications to learners about upcoming assessments if the assessments are within the platform.
7. Unified data and reporting
Delivering assessments in the LMS unifies data and reporting capabilities, which means all learning and assessment data is stored in one place. With a separate assessment management system, high level information is passed to the LMS, but more detailed information is held in the AMS. If using just an LMS, it may be easier to track and analyze learner performance, allowing for more comprehensive insights and easier compliance reporting, but it does depend on how much assessment data the LMS gathers.
Pros of using an LMS + AMS for assessments
And here are the advantages of a separate AMS to work alongside the LMS:
1. A wider range of assessment and question types
A key advantage of an AMS is that it will allow a wide range of question and assessment styles, which can let you measure skills more effectively (and more validly and reliably). There will typically be 20+ question types and the ability to deliver performance exams (e.g. where someone performs an IT-related task rather than answers questions on it.) It will also be possible to have a randomized selection of questions, branching, and other capabilities. These capabilities assist you to assess more authentically and accurately.
2. Item bank allowing question re-use
A distinguishing feature of an AMS is that it has an item bank, usually broken down hierarchically by topic and allowing meta tagging. This means that questions can be written once, organized by topic, and then used in multiple tests, which can save a lot of time. This is often not possible in an LMS, which associates questions only with the test they are in.
Item banks also allow selecting questions in a test randomly by topic or meta tag. If you have hundreds or thousands of questions, an item bank will make your internal processes much more efficient.
3. Versioning and review of items
A good quality question authoring process lets subject matter experts create questions that then go through a review process prior to finalizing. LMSs let you create and edit questions, but many do not permit a structured review process, nor do they version questions. Questions that are properly reviewed are more likely to result in good-quality tests and good-quality decisions.
4. Audit trail and legal defensibility
If you are using assessments in situations where decisions on people (hire, fire, promote, etc.), then you need to be able to defend those decisions, perhaps even in a court of law. An AMS will have audit trails, both for the authoring process and the test-taking process so that if there is any dispute, evidence is available as to what happened. This also gives more reliable data when test results are being used to show regulatory compliance.
5. Differential security
For many organizations, it’s important that there is access control for different parts of the organization. So, for example, one department can author questions for their department, but not see the questions or results for another department. An AMS will usually have such differential security controlling the item bank and test results.
6. Reduced cheating and test fraud
Cheating at workplace tests is an increasing challenge. Some people take screenshots and share questions with each other or otherwise cheat at compliance and other internal tests. This has sometimes (for example here) resulted in regulatory fines, and of course, reduces the effectiveness of tests. LMSs tend to be built for easy access to learning, whereas AMSs have capabilities to reduce test fraud. For example, an AMS will allow shuffling or randomizing of questions so that it’s harder to copy someone else’s answers and will have a lockdown browser that makes it harder to cheat during a test. For those with higher stakes needs, an AMS will also offer the ability to have proctoring, either automated or from a live online proctor, which significantly improves security.
7. Robustness and scalability
It’s important for most tests to be robust, so that if someone’s equipment or connection fails, the test results are still saved, and the test can be resumed. An AMS is likely to do this more robustly than an LMS. An AMS will also usually be more scalable. Sometimes it’s needed to deliver a large amount of tests in a short time period, e.g. when a new product is launched or a compliance deadline is near. An AMS can typically handle a larger assessment load than an LMS, as it’s built to manage the load of thousands of people taking a test at the same time.
8. Printed tests possible
Sometimes it’s useful to be able to print tests, for example, for people without online access or as a backup. If you need the ability to print a test and scan the results back to combine in a single data result set, this usually needs an AMS.
9. Fairer, more inclusive tests
There is increasing understanding that tests need to be designed in a fair way to make them accessible and valid for all minority groups. For example, you have to consider fairness to those with accessibility needs, those from different cultures, and people whose native language is not the language of the test. Although good LMSs seek to be fair in presenting learning, they are less capable at assessing it. AMSs typically have capabilities that help make tests more inclusive, for example, very strong support for accessibility, text-to-speech capabilities, and the ability to offer non-English speakers on-the-fly translations of tests. Their review capabilities also help you set up a process for the review of questions to reduce unfairness to some cultural or demographic groups.
10. Better assessment-specific reports
An AMS will typically have a much better range of reports about the meaning of assessment results, and will help you interpret them and take action more effectively. It will also have psychometric reports like an item analysis report which helps you identify weak questions and improve the quality of your tests. If you are using assessments in circumstances where trust in the results is important, these reports are vital.
The diagram below shows a typical item analysis report. The questions at the bottom right (in red and orange) need review as their inclusion may be impacting negatively the quality of the test. If you are using tests for any serious purpose, it’s crucial to run such reports.
11. Support internal certification
There is an increasing trend to set up internal certification programs, whereby employees can obtain accreditation for skills and capabilities within the organization.
These are easier to organize within an AMS with its security capabilities, optional proctoring and digital badges.
12. Easier to translate assessments
An assessment management system will let you manage the translation of questions and assessments, so that you can create and deliver the same test in multiple languages and collate and interpret the results, typically via a tool like a Translation Management System (TMS).
13. Better decisions
This is where it all adds up. If you are using tests for an important purpose, e.g., to meet critical compliance requirements or to identify who to promote, then you need to be able to rely on the results. Tests must be reliable, valid, fair and secure. Using an LMS alongside an AMS helps create and deliver high quality assessments that can prevent mistakes and increase performance, which brings huge business benefit.
Summary
So, what is the right answer for your organization? It will depend both on your needs and also on the capabilities of your LMS. To help, the below table is a brief overview of the functionalities available when using a LMS or an LMS alongside an AMS, though it’s worth noting that some LMSs are much stronger than others in their inbuilt assessment capabilities.
| LMS alone | LMS plus AMS | |
|---|---|---|
| Question types | Limited | Wide range |
| Item bank allowing question re-use | Usually no | Yes |
| Printed tests | Usually no | Yes |
| Observational assessments | Usually no | Yes |
| Can translate assessments | Usually no | Yes |
| Protection vs cheating | Basic | More capable |
| Suitable for quizzes and low stakes tests | Yes | Yes |
| Suitable for medium/higher stakes tests | No | Yes |
| Integration needed | No | Yes |
| User interface | One system | Two systems |
| Cost | LMS only | LMS + AMS |
| Data stored | All within LMS | LMS + AMS |
In many ways, an LMS is good for managing the learning journey, but if there is a measurement or assessment part of the journey, an AMS will manage that part of the user experience and data collection much better. And if you are making decisions about people as a result of their test results, an AMS will give you better and more trustworthy information. Due to these differing strengths of each solution, we recommend a combined approach, where you elevate your LMS capability with the rich assessment and reporting functionalities of an AMS.