Case Study: Using Diagnostic Assessments to Improve a Government Agency’s Workforce
The Aurelius Group (TAG) provides Federal acquisition, human capital, and technology consulting to private industry, federal agencies, and the U.S. Department of Defense. The company, a verified Service Disabled Veteran Owned Small Business (SDVOSB), creates progressive, end-to-end human capital solutions for its clients. These initiatives include customized employee assessment programs.
Fortunately, the Federal Acquisition Institute’s (FAI) Federal Acquisition Certification (FAC) Program provided a framework for focusing on key positions and competencies. Based on this framework TAG designed an assessment program to enhance leadership’s awareness of the strengths and weaknesses of the FAC population and to improve the efficiency with which training and workforce development dollars were spent.
In creating a program to meet the agency’s needs, the firm combined its three core competencies—acquisition expertise, strategic human capital projects, and technology consulting — to create a unique assessment solution. At the program’s outset TAG had two primary goals. First, the team developed the business rules that would allow the addition of assessments to seamlessly integrate with their client’s current workforce development framework. Second, TAG needed to evaluate and select a flexible technology platform capable of supporting those business rules and any changes that might be made to them in the future.
The project team analyzed about 50 testing and assessment systems, from which Questionmark emerged as the most comprehensive platform for measuring employee competencies.
“We pulled Questionmark off the shelf and were able to start giving assessments a few months later in a fairly customized way,” recalls Aurelius Group Chief Information Officer Justin Devine. “We’ve been able to adapt our assessments to changes in the program, and we’ve worked with Questionmark to create a system that matches our client’s needs. Objective assessments provide clear, measurable information about people’s skill levels and highlight areas where they need to improve their knowledge and skills. Test results also help us monitor the success of the program and continuously fine-tune it.”
Questionmark assessments provide annual or biannual snapshots that show how much employees know about the complex bodies of knowledge their work requires and identify competency gaps that can be addressed through further learning. The assessments use randomized questions from large item banks pertaining to three different disciplines: Federal Acquisition Certification (FAC) for Contracting, Contracting Officer’s Representative, and Program Project Managers. After completing an assessment, each participants immediately receives targeted training recommendations that feed into his or her Individual Development Plan (IDP).
Assessments are administered using Questionmark via virtual servers in the Cloud. This makes them available to registered users anywhere on the internet. Employees may take them from anywhere they wish. Since they need to look up information on the job, they are allowed to do the same when they take these “open-book” assessments.
Careful attention to reporting and analytics help the project team provide data-driven management, planning, and decision-making.
“We evolve the content of the program based on data gleaned from tests and on what analytics tell us about the test results,” explains Justin. “In addition to what we gain from the technology, we do deep reviews of our competency modeling and competency assessment programs to make sure we stay on the right track.”
In the future, the team hopes to use the Questionmark Gap Report to compare groups of participants with each other to determine differences achieved at the question, topic or assessment level — a means of tracking and analyzing performance over time. Also in view is the use of Questionmark’s Results API, which uses OData to collect results and enables dynamic, customized reporting via business intelligence tools.
“We’ve been developing some proof of concept dashboards using OData and Microsoft’s PowerBI suite and we’re really excited about the early results. We’ve decided to make reporting and analytics based on assessment results a core competency for our firm.”
The program includes Continuous Learning Points (CLPs) for assessment participation, providing an incentive for people to be aware of and build upon their knowledge, skills and abilities. TAG has created a custom extension to Questionmark that automatically generates completion certificates and sends them as email attachments after people successfully complete assessments.
“This gives people tangible proof that they use to claim the continuous learning points they earn to maintain their FAC certifications,” explains Justin. “It provides evidence that they did indeed complete the assessment. Also, the process of taking the assessment is valuable in and of itself, since responding to questions has been shown to enhance learning.”
Assessments have contributed greatly to the success of the workforce improvement program, which reveals the aggregate strengths and weaknesses of the workforce and enables the client to direct resources to high-value development opportunities. Trend data gleaned from the assessments demonstrates decline and improvement over time and provides objective support for training resource requests.