Welcome to this Questionmark podcast. Questionmark podcasts bring you news, ideas, and advice about assessments and learning.
Joan Phaup, Questionmark: This is Joan Phaup from Questionmark, and I’m talking with Adam Zaller, director of Services University of NCR. Hi Adam.
Adam Zaller, University of NCR: How are you, Joan?
Joan: Good. It’s great to have you here. Could you just tell us a little bit about yourself and your organization?
Adam: NCR is a worldwide company. We do business in more than 150 countries, and it’s self-service technology, so ATMs, self-checkout machines, kiosks in your airport or at your car rentals, as well as in your grocery store and other places. So a worldwide company with a very large break fix(?0:45) services organization of about 13,000 associates, and my group, Services University, helps to educate and train those individual to perform their jobs, so I have 44 people in about 20 different countries, and our whole focus is to educate as well as then through life cycle education certify them in their skills as they move forward and progress in their career.
Joan: And I understand you have a global skills-based certification program. I’d like to hear a little bit about that.
Adam: The program started last year. It was very well received, and has grown exponentially. It is a certification program that’s designed to certify the skills of our customer engineers, what we call CEs, in our field. They’re the folks that actually fix our equipment for the customer. So we really want to make sure that they get it right the first time, that they’re capable at a basic and advanced and expert and a master level, so that’s the progression we have in our certification world, and so far to date, over 1100 CEs and just, actually just seven months have been certified in over 45 countries. We have been on a pace of almost doubling quarter over quarter, and we’re really looking forward to what our third quarter of certification brings us. But so far, the program has been fantastically well received. I think one of the reasons why is because through the design and development and implementation we’ve done two main things. The first is that we brought our engineers in, our customer engineers in to help us design and build a program, even write some of the items, as well as we’ve had a very large internal marketing campaign to bring the awareness out for these 13,000 individuals.
Joan: Could you talk a little bit more about that and how you involve the people who are actually being certified in the content of your assessments?
Adam: Yeah, I think that’s really been the hallmark of the program. One of the ways in which we have designed the program is we brought the CEs in, and we asked them, what would be the constructs, what would be the tasks? The subtasks? What type of design would we have for the program? So we really engaged a global audience. We used over 100 customer engineers to help us design and build the program. The philosophy was that if, who better to build and help us build the program than individuals that would be using it? And then they would be the early adopters, and those early adopters would then spread the word to everybody else about the program and its value. So from day one, they’ve been very influential in the program. We still use about 100 or so customer engineers globally, and in a good 20 or 30 different countries to help us do the item writing. That has been a great advantage for us, because we’ve really been able to make sure that our questions are of a global perspective, and so that we’re not held to a US or America’s perspective, and that really provides validity for the overall test. And since it’s done by customer engineers, it’s more easily accepted as a valid test by them.
Joan: I’m curious to know how you manage all that with all these questions coming from all over the world. How do you manage that within Questionmark Perception?
Adam: We couldn’t really do this program without Questionmark. I’m really excited though for version five. Just seeing some of the things I’ve seen today, I think it’s going to be a huge help for us. Today we use a jump page for all the different languages, because we have a lot of languages. So after this jump page, then we go and we look at different, and they pick or select the language, and then based on that, they get the items for that language. So we basically create this certification seven times over. So the new version five will be wonderful, because we’ll have one test with each item having multiple entries or capabilities for language based on the language that you choose. So from what I’ve seen today, our test itself will be much simpler from an administrative standpoint, which will be fantastic, but even with that said, today, Questionmark, when we looked at the tools, was really the only tool that could help us organize this very large test, or actually it’s tests, into easy administrative functions.
Joan: I’m just curious right now, how do you collect the questions from the CEs?
Adam: Right now, it’s done through workshops. We have learning consultants around the world hold workshops where they might be, so if we’re in Thailand and we have a program going on, we’ll pick from some of the individuals who are training some of the more experienced as well as some of the more borderline folks, and we’ll have a workshop, and item writing workshop that we’ll conduct with them where we teach them how to write items. That’s really one of the most important things when we had consultants come in to help us with the initial design. They taught us about item writing, so we had, we do an item writing workshop, and then they will provide items at that point in time, but then we also provide them a conduit to give us other items as they think about them. And right now, a lot of that’s done in soft copy, which again, when I look at Questionmark Live, and the ability to use that portal for item writing, I think that will be fantastic as well. So we’re very interested in looking at that for the future, because I think we’ll be able to get away from a lot of the soft copy and having international courier of our questions back over to our administrative staff, which is in the US.
Joan: Do you have any particular advice for people who want to do certifications? Any bit of best practice you can share?
Adam: I think it really boils down to just a couple. The first is senior management involvement and engagement, and make sure that senior management is on board. For us it was great from the very opening. This was an idea that was brought forward by our CEO, as well as our Senior Vice President of Services, so to have those two on board, engaged, and wanting to move this forward was something that you don’t get to see too often, and when you do, it’s something that you really want to take advantage of. So that was one. The other one was, when creating a valid skills-based certification, bring in the experts to help you design it. So we brought in some of the best consultants that you could have to help us really make sure that this was a valid certification. It eased our legal team’s fears and concerns, as well as it made sure that we were able to articulate that we had a very top-notch, robust certification program, and that we were following the correct rules and regulations. The second, or the last one that I would really say, or one of the last ones I would say is, bring in your end users to help you design, build, and implement. They’re the ones who are going to make sure that you’re successful, and they’re going to provide you the perspective that you need to succeed, as well as help you write the right items for the right job tasks and the right subtasks. And then finally, market, market, market, market, and market. Just because you build it doesn’t mean that they will come. So we’ve done a lot of internal marketing. It doesn’t have to be exhaustive as far as dollar. But it does have to be very visual and visible, and so we’ve done a very solid program, a marketing program where every individual gets contacted six to nine times a quarter based on their past usage of the certification program, and a reminder that they have certifications available for them to take, and the value of attaining that certification. So I think those are the best practices that we’ve used that have really helped us gain not only a strong foot-hold, but this program has continued to become more advanced and continued to gain momentum every single day.
Joan: Great. Thanks so much.
Adam: No problem. Thank you very much. This was a lot of fun. |
