John Kleeman welcomes Sonny Yuen, Senior Director of Product Strategy at Workday, to discuss the value of assessments in the world of e-learning and corporate learning. Sonny’s impressive background includes roles at major e-learning companies, and he shares insights into the field.
Together, they delve into the significance of assessments in the learning process and how assessments are integral to verifying competence and capabilities. The discussion covers Questionmark’s integration with Workday, AI’s role in learning, recommendations for setting up successful learning programs, and the importance of diversity, equity, and inclusion in learning.
Sonny emphasizes the need to involve stakeholders and suggests that organizations allocate dedicated time for self-elective learning to foster professional development. Listen now to explore the evolving landscape of corporate learning and how technology plays a crucial role in optimizing the learning experience.
Full Transcript
John Kleeman:
Hello everyone, and welcome to Unlocking the Potential of Assessments, the show that delves into creating, delivering, and reporting on fair and reliable assessments. In each episode, we chat with assessment luminaries, influencers, subject matter experts, and customers to discover and examine the latest and best practice guidance for all things’ assessment. I’m your host, John Kleeman, founder of Questionmark and EVP at Learnosity, the assessment technology company.
Today, we’re really pleased to welcome a genuine expert from the world of E-learning and corporate learning. Sonny Yuen is senior director of product strategy at Workday and is responsible for the strategy of Workday’s learning management system, Workday Learning. Sonny has a really impressive background in the space, having worked at several of the major e-learning companies in their formative years, as well as his current role at Workday. He’s worked in senior positions at Cornerstone OnDemand, Schoolzilla, Achievers, SuccessFactors, and Udemy. Sonny, very nice to have you with us today.
Sonny Yuen:
Thank you, John. It’s a pleasure to be with you.
John Kleeman:
So let me ask you the question I ask everybody, how did you get into the learning and assessment world? What started you off on it?
Sonny Yuen:
Yeah, it’s a great question. So initially, I fell into people technology with SuccessFactors, and from there, got exposure to corporate enterprise learning and just really connected with it. Given all of us work, all of us need developmental or compliance learning. It just never stops and the interest sparked there.
John Kleeman:
And talk us through a little bit about your career prior to Workday. You’ve been a lot of very interesting companies, really almost elite companies in the space. Talk us through it little bit.
Sonny Yuen:
Sure. Like a lot of folks in software technology, I got the startup bug. So a number of those stops were in formative years, right? Either early pre-IPO, through the IPO phase, and rapid growth phases for those companies. And I did that for quite some time and had a unique experience in that. At three of those stops, in my role in partnerships and alliances, I actually created partnerships with Workday. So I had great familiarity with Workday prior to joining because of those partnership relationships that we built in the technology integrated solutions.
John Kleeman:
And what did you learn at these different companies? Is there any theme or something that’s worth sharing with other people about similarities or what they’re all a bit like?
Sonny Yuen:
Yeah, I think I mentioned at the outset these solutions are interesting to me because they touch all workers. Unlike a sales customer relationship management, it’s a specific audience of worker that engages with that technology and people systems and particularly learning touches everybody. So the opportunity for impact is what’s most exciting, and I saw that at every stop along the way, whether it’s the organizational culture and capability being developed through learning and development p to how customers are trying to be successful, implement their strategies with the capabilities they need to be successful. Every step along the way has reinforced that point of view.
John Kleeman:
And I imagine most people who listen to this will know a bit about Workday, but some people might not. So can you introduce Workday? Tell us a little bit about who they are?
Sonny Yuen:
Yeah, absolutely. Workday was founded in 2005. We’re a cloud-based software company that’s a proven leader in HR finance software. And our legacy were the founders of PeopleSoft who founded and grew that to the global business that it was.
John Kleeman:
So Workday does financial human capital. And tell us a little bit about the learning strategy. What is Workday’s learning product strategy?
Sonny Yuen:
Yeah, roughly seven years ago, we recognized, as the core HCM and talent management capabilities had matured, that learning was the natural next step. A few reasons for that, we think that being the source of truth for real-time worker attributes allow us to identify an audience based on, say, a skill gap and then target learning interventions to them to fill that gap. And so we thought it was a natural fit for where the company was and what our customers were asking. And so now we’ve got a seven-year-old product with roughly 2,600 customers worldwide spanning industries for learning.
John Kleeman:
And what’s the key value of learning in the enterprise?
Sonny Yuen:
So from our point of view, there’s two things that get combined into our mission, which is powering workforce readiness. The first is the stuff we have to do to operate safely and legally, and then importantly, it’s how do we develop the skills and capabilities so workers can be successful in the job they have today and prepare for what comes next.
John Kleeman:
And so this is an assessment podcast. Where do assessments fit in?
Sonny Yuen:
That’s one of my takeaways from working and learning across enterprise K-12, primary education, higher education, life-long learning. You cannot escape assessment in some form, right? It’s the most common accepted tool we have to verify some level of competence, knowledge or capability.
John Kleeman:
Yeah, yeah. Where do assessments fit into the Workday Learning or the Workday more wider ecosystem?
Sonny Yuen:
Yeah, that’s a great question. So obviously with straight learning for compliance or skill development, assessment plays a key role. We partnered with you, the Questionmark, obviously for our customers that have very extensive needs to analyze or create a variety of interaction types to really measure and verify knowledge and capability and every customer has assessment need. And actually as we move to more skill-based strategies, I think the umbrella of assessments only grows larger to include classic testing capabilities, practice participation, assessment by managers, peers, customers, things of that nature. And so tools and capabilities that are best in breed to market are going to be even higher demand in our customer base would expect.
John Kleeman:
So do you want to just explain about what you mean by skills in the Workday context, you’re talking about technical skills, other kinds of skills?
Sonny Yuen:
Sure, yeah. Skills span the gamut from what you could consider sort of professional, technical, hard skills to working… Or professional soft skills, right? Things that are personality, behavior, trait, developmental items versus hard. I know how to work with SQL-type technical skills.
John Kleeman:
So is there a idea in Workday Learning that you can capture the skills of the entire workforce and then make decisions based on that? Or is it more focused on individual learning journeys or…?
Sonny Yuen:
Yeah, actually, the skills are an absolute foundational element of all of Workday’s human resource technology. So we have what’s called the skills cloud. I think we were very early if not first to market with that, which is our ontology, which is the sort of baseline or the Rosetta Stone for how we connect all these people services through skills. And the key thing with that is that’s the foundation for machine learning. So we make skill-based course recommendations in the context of learning. You tell us what you’re interested in learning, we can make recommendations based on that. We also do things like infer what a worker’s skills are based on time and role feedback, course completions, even gig opportunities they participated in. So this starts to feed this virtuous cycle and enables us to help organizations at scale, identify the skills in the organization, develop them as well.
John Kleeman:
And when you have a skill, are there also levels in the skill? How do you deal with the fact? I mean, for example, say your skill is sort of presenting or using PowerPoint or something like that. Is there sort of concept of different levels of expertise? How does that work?
Sonny Yuen:
Yeah, absolutely. So skill level or proficiency is sort of the very first question you get from customers when you start talking about implementing skills as part of their overall strategy. And the key thing for vendors like us, and I would imagine Questionmark is allowing flexibility for the customer because a high-proficiency PowerPoint user on Workday may look different than a high-proficiency PowerPoint worker skill-haver in Questionmark. So that customers need to have that flexibility to define the level of proficiency for their organization. And even within their organization, you’ll see a variety across departments, divisions, even managers. So we’ve got to create tools that can account for that and help organizations sort of fine-tune what those things.
John Kleeman:
So for example, an expert in PowerPoint to, say, a consulting company where they use PowerPoints all the time might be a different level of expertise to say an expert in a company that the presentations are not so important.
Sonny Yuen:
That’s right. That’s right.
John Kleeman:
So talk us through the different kinds of assessments. So some of it presumably is self-assessment or a 360 assessment, and some of it is the more sort of conventional assessments of where you ask people questions or observe them. How does it fit into the learning journey?
Sonny Yuen:
Yeah, really, it’s a mix depending on the context. And in some cases, it can all of those things that verify a capability within a worker. So it can be completing a digital course and taking that digital assessment, the sort of classic test-type example. But I also need to be observed by my manager or an expert who’s authorized to observe me and verify I can actually perform that work. And then beyond that, it may also include time and role and ultimately sort of 360 feedback, right? So there’s just four elements right there that can be what make up the validity for my skill.
John Kleeman:
Yeah. Yep, yep. And where does sort of question assessments fit into that?
Sonny Yuen:
Yeah, it’s a great question. So the majority of our customers build sort of classic digital assessments and their score monitoring tools and what have you. We’ve got some video interaction capability in the native video platform. But for those particularly we find in what I like to consider high-risk compliance industries, financial services, sort of high end manufacturing professional business services, they’ve got much greater need to test and verify in different ways and then analyze both the results they’re capturing from those assessments and how effective the assessments are. And that’s the capability that we don’t offer today, and we look to partners like Questionmark to deliver.
John Kleeman:
No, I understand. And why is Workday Learning different from other LMSs? There are a lot of LMSs out there. What are the distinguishing features?
Sonny Yuen:
Yeah, there’s two things really and they’re related. So the fact that we are built into the core Workday environment means we always have the most up-to-date, real-time information on worker attributes at what are attributes. It’s time and role feedback, organizational hierarchy. Everyone has that. Usually, they’re pulling from us, but it’s a point in time. But through that, we’re able to, like I said earlier, identify audiences. We can analyze the skills’ portfolio of the organization and identify people that have gaps or maybe where we want them to level up and then target learning interventions in the form of a curriculum or programme, push that to those folks, and then measuring the backend, how effective it was. So that’s what a key foundational benefit of Workday Learning.
The second is that in these other talent contexts, we can surface recommendations sort of in two directions. So as workers are starting to manage their career, look at that next job that’s up the chain or adjacent to me, they can map their skills to the profile expected, and then we can make learning recommendations right there in that context of where they can develop where their weaker points are or prepare better. It works the other way as well as you match to, say, a gig opportunity, a short-term work project, sort of out of the boundaries of your organizational structure, it requires certain skills. You’ve got a certain skill profile, there may be a little gap there. We’ll make learning recommendations there so you could be better prepared to succeed in that gig participation. And so that ability to meet people at the point in time in context is really tough to do. And we think we’re well subjected to do that because all this capability lives within the walls of Workday.
John Kleeman:
So essentially it’s the real-time source of truth about people in real time in your organization.
Sonny Yuen:
That’s right.
John Kleeman:
Yeah. Yeah. And I imagine that assessments feed into that in real time. So somebody takes an assessment in Questionmark or another system, that data gets populated in the Workday system in real time as well.
Sonny Yuen:
That’s right. That’s absolutely right.
John Kleeman:
Yeah, yeah, yeah. And by the way, I should mention that I was surprised that, at the beginning of the call, Sonny is standing up for this interview and I believe you’ve taken a few calls standing up. Can you just talk us through that?
Sonny Yuen:
Yeah, sure thing. So in the COVID times, I was setting up my home office and I thought I should get one of these automatic desks that goes up and down. We had those at the office, and I found when I’m talking to customers or partners like yourself in high-stakes conversations, my mind works better standing up. And so I’m up and down throughout the day sitting and standing through my Workday.
John Kleeman:
It’s really interesting. I think I might try. So I think one of the things we talked about when we were preparing for this call was that quite a few, Workday Learning users are desk-less people are manufacturing or hospitality. And I think that’s one of the capabilities of Workday Learning to handle that. Is that right? And how do you manage learning for people who don’t have a desk or sometimes don’t have a computer?
Sonny Yuen:
Yeah, that’s a great question. So we actually have been diving deeper into this topic over the last year and change our recently concluded learning customer advisory council was focused squarely on desk-less learners. And there was a couple of challenges we have there. The primary one being they’re not in front of a laptop. Some of them may not have access to smart devices or access to Workday on their personal device or company issue device. So how do we bridge that gap? And so we’ve been working closely with these customers to support sort of a hybrid approach that sometimes leverages partners for some of these things that have tools specific to those types of workers.
But again, the important thing for us is to bring it all back home to Workday. So from an organization point of view or leadership point of view, you make sure you have the real full picture and there may be a variety of ways that these desks workers engage with either Workday Learning, coming to a classroom, engaging through a kiosk, maybe through their mobile device and similarly for the other tools in the learning ecosystem like Questionmark. So we are working through this. We have some ideas that we’re excited to explore, particularly around what role the manager plays in learning recommendations for desk-less learners where they don’t often come into the system, so we can’t present them some of these recommendations. So how do we leverage the manager who is that channel for those workers? So that’s some area we’re very excited to explore in the coming months.
John Kleeman:
And I know we have some work going on with some of our joint customers in areas of observational assessment where in practical tasks, people are observing workers and then getting the results put back into Workday. And I guess, that’s another use case in that kind of space.
Sonny Yuen:
Yeah, I think actually just this week, we had a meeting talking about our healthcare customers and where we can partner to better support that.
John Kleeman:
Great, great, great. So look, AI is the word of the moment, and I’m sure Workday’s been doing stuff in AI for a lot more than just recent times. What is Workday doing in AI and how does that affect Workday Learning use cases there?
Sonny Yuen:
Yeah, that’s a great question. So I mentioned earlier the skills cloud, right? The skills cloud, I think we announced eight years ago, seven or eight years ago. That’s been native capability building off of large language models in machine learning capabilities since then, so this is not something new. Now, there’s certainly new applications like generative AI that are at the top of mind right now peaking in the hype cycle. We’re looking at where the best opportunities are to leverage that capability in ways that meet the requirements we have with our customers. And what I mean by that is Workday has a lot of very secure information, whether it be payroll, PII across your workers, financial data, things of that nature. So that’s out of the gate commitment we have to our customers to make sure that is secured at the highest levels. And so there’s working through what can you share with these types of GenAI models that protects that and respects that security and also provides some results.
Additionally, some of the ideas I’ve seen of late in learning with respect to GenAI are things that we were doing nine years ago, but calling it something different. So for example, at a past stop, one of the best demoing features we had was around performance feedback and suggesting language based on the ranking. Customers loved it when you demoed it. Once you got into the weeds of it though and tried to use it, this just fell down. You ended up creating custom feedback anyway. So I see some similar things could be happening now and we want to be smart around, again, respecting the security in privacy of our customer data. Also in the learning context, I’ve heard some use cases of folks going out scraping the web or scraping content vendor catalogues and creating outlines of content. There are some potentially sticky IP-related issues there that need to be resolved. So we’re taking a cautious approach to GenAI, but we have long been leveraging machine learning and large language models in the services we deliver.
John Kleeman:
And could we perhaps drill down a little bit into the recommendations? And the way you use the data potentially with AI, it seems that might be very interesting to people that, essentially, you can take the results of the assessments and other things and then make… Would you go as far as to make predictions about people or is it more just determinations about people and what skills they’ve got or what skills they might want to learn?
Sonny Yuen:
Yeah, that’s a good question. So I don’t know if prediction is the right use for our application, but certainly our recommendations are based off of what a worker tells us they want to develop. And then as they interact more with the system, we start inferring more about their skill capability and fine-tuning those learning recommendations.
John Kleeman:
And does it also work in the terms of evaluating the quality of learning and effectiveness of learning? Do you identify learning that’s more effective than some other learning, or is that not in scope?
Sonny Yuen:
That’s an interesting concept that is not something we’re focused on at present.
John Kleeman:
And you’ve been involved in corporate learning in a lot of different places. How is it going to change in the future? I mean, hard question perhaps, but equally, I promise not to come back in five years and check if you are right.
Sonny Yuen:
Sure. So first I would like to be hopeful for all of our L&D friends out here and say the focus is going to be a topic of investor analyst day calls and extra funding, but that’s unlikely to be the case. I think the holy grail for all of us continues to be measuring the impact on the business of learning programs deployment. The compliance end is, that’s easy to measure relatively speaking, but it’s again identifying, “Hey, we’re shifting our strategy here.” As an organization, we need people to add these skills we don’t have yet. We invest in programs to build these skills. We can test and assess that workers have captured and maybe are utilizing some level of expertise gained from those programs. Is that affecting accounting for other factors, the output? Can we tie these investments to business performance directly? And I think this is, like I said, been the holy grail and few have nailed this yet for a bunch of good reasons, but I think that’s still the dream.
John Kleeman:
So sort of the level four of the Kirkpatrick or sometimes I think level five that Phillips brings in or whatever. Obviously without naming names and things, do you have customers who are able to do that with Workday Learning and able to identify the business impact of their learning investments?
Sonny Yuen:
We do Workday, what we call Workday and Workday are wow, our internal deployment and processes. I think they’re very good at this and we leverage that story with our customers quite a lot. And certainly there are a number that are very good at measuring the effectiveness of programs to do Workday Learning. I’ll say one other thing though related to this is we’re still, I believe, in the early days of this proliferation of content services. So these are things, like I say services because LinkedIn Learnings, the Udemy’s classic content services, are sort of on demand. They have a great experience the way their tools work. You need to engage with it there. Other things are evolving now too, whether it be AR/VR, which for a few years have been trying to find its footing and who knows what’s to come. Maybe it’s like a more gaming like interaction.
This stuff is in the early days. And so it’s critically important for us at Workday to make sure that all these things that work, which are fantastic for the job they do work for our customers within the ecosystem with Workday as the center, the anchor around this stuff. So we’re not going to ever own everything in your learning environment, nor do we want to make sure that everything works for you and importantly for the learner, because we hear people talk about learner experience. And I think the shorthand for that is having a nice UI with good machine learning recommendations of content, but I think it’s really owned by the worker because they’re going to enter the context of needing learning from multiple different ways. And so our job is to make sure wherever they enter, because they own the experience, they have the most real-time best recommendations and data possible. And so it’s a little bit flipped, right? It’s like I think any provider that says they are owning the experience of the worker is stretching it a bit because the worker owns it.
John Kleeman:
So it sounds like there’s really talking about two angles here. One is from the organization, get the business value of the learning and drive it that way. And the other is from the individual learner, empower the learner to learn what they need to learn.
Sonny Yuen:
That’s right.
John Kleeman:
And assessments, whether they fit into that, I guess it’s partly sort of identifying the skills of the learner and so what they need to do next. And also maybe from the organization identifying what people have learned. Is that fair or where would you see assessments fitting in?
Sonny Yuen:
Yeah, absolutely. I mean, the classic example is from the organization’s point of view. We need some standard way or common language to trust that we’re meeting some objective of learning. Assessment is the key to that. And from the learner’s perspective, it’s twofold. One for them to understand, “Hey, I took this thing, I can understand migrating or proficiency or position based on an assessment that if my colleague took it, we could measure results effectively.” And then that drives what comes next. There may be a gate that I’ve got a pass to move on to more advanced capability or it changes the recommendations of what I get presented next.
John Kleeman:
And what do you think if somebody is looking to set up a learning program or something that involves assessment and learning, what do you think is the key success criteria or key things they need to try and do, whether they’re using whichever system or software they’re using, any sort of advice you’d give from all the experience you’ve seen out there?
Sonny Yuen:
That’s a great question. I think the one bit of advice would be take your time, don’t rush and make sure you get the perspective of your key constituents, and not just their perspective, but their participation in crafting and developing the content flow or the curriculum as well as what the assessments should look like for what their objectives are.
John Kleeman:
So speak to the people who need the output, people who actually are going to be learning, people who rely on it, all the different stakeholders.
Sonny Yuen:
That’s right. And multiple of them, right? One engineering director may have a different perspective on the required skills and proficiency and how you measure that versus another within the same organization. So make sure you count for at least some variation there and gain agreement before you put things to stone.
John Kleeman:
So a key way to be successful is to speak to a lot of stakeholders in advance and make sure that you are bringing them along with you in the journey rather than, say, just leaping out that you’ve got a dream or a vision and making it happen, ignoring people.
Sonny Yuen:
That’s right. And don’t talk to everybody, but a worthy sample size.
John Kleeman:
What about diversity and inclusion and learning? Is that a factor, accessibility, cultural factors, is that a factor that I’m sure it must be really important to lots of your customers?
Sonny Yuen:
Absolutely. And the interesting thing about the DEI tied to learning is it’s always existed there. This has always been fundamental to learning and the delivery and coaching and development of our teams to think with broader perspectives. So I think learning has always been foundational to any type of DEI initiative or effort.
John Kleeman:
Yeah, Yeah, yeah. No, that’s a great point. And so what is it that gets you up in the morning? What is it that you enjoy doing, and why are you in this job and what do you want to make a difference in the future? Sorry, lots of questions there.
Sonny Yuen:
That’s a great question. Yeah. Yeah. There are a lot. I’ve been really blessed in my professional career and the most exciting thing for me is to work with the brands that I recognize throughout my life, the things that we all interact with, whether they’re the consumer goods we use or the services or the technology products we have, and hear how they’re solving these problems or what they’re challenged with. And then be able to play a role there either as a consultative partner or someone who can connect the dots across multiple customers with similar challenges that we can then solution for as a product and service organization. So that really excites me, right? I love looking at my calendar and seeing the logos and the brand names of folks I get there.
John Kleeman:
Well, at Questionmark, we are very pleased to be a Workday partner and to help those organizations with their assessment challenges. And it’s a great organization to work with, so thank you.
Sonny Yuen:
Absolutely. And again, this is my second time around partnering with Questionmark going back to my Cornerstone Day. So excited to still be partnering.
John Kleeman:
Well, we have a lot of customers who use Cornerstone too, but we’re very excited to work with Workday. So any final comments or suggestions to people? I mean, we talked about getting stakeholder view. Anything else you want to share as a takeaway?
Sonny Yuen:
Yeah, one thing I’ll say for all of our learning and development friends out there, the executive buy-in goes without saying is very important. But one key thing I think we can all do, which doesn’t really cost our organizations too much is get leadership, buy into clear space for learning of the self-elective form. So what I mean by that is even if it’s an hour a month, clear space, where no meetings are allowed as at an organizational level for your workers to engage with professional learning that suits them, whether it’s I want to be better at managing my calendar, I can take content on that, right? I can get consume courses. But that, I think, would be the biggest thing to try and do is get leadership buy-in from the very top to clear dedicated time for learning that is not sort of required compliance learning.
John Kleeman:
That’s really interesting. And I think sometimes it’s very hard to get the important done when you’ve got the urgent happening all the time. Thank you, Sonny. I really appreciate talking to you and obviously if anybody wants information on Workday Learning, it’s workday.com and there’s lots of information there.
So thank you everybody for listening with us today. We appreciate your support. And don’t forget if you enjoyed this podcast, why not follow us through your favorite listening platform? Also, please reach out to me directly at john@questionmark.com with questions, comments, or if you’d like to keep the conversation going. And you can also visit the Questionmark website at www.questionmark.com to register from any of our many best practice webinars we host monthly. Thanks again and please tune in for another exciting podcast discussion. We’ll be releasing shortly.
Well, it was a pleasure. Thank you everyone for listening for us today. We appreciate your support. And don’t forget, if you’ve enjoyed this podcast, why not follow through your favorite listening platform? Also, please reach out to me directly at john.kleeman@learnosity.com with any questions, comments, or if you’d like to keep the conversation going, and I’m sure John Weiner would also welcome you to reach out to him if you’re interested in a dialogue with him. Please check out the www.questionmark.com and learnosity.com websites for more information or to register for our webinars. Thanks again and please tune in for another exciting podcast discussion we’ll be releasing shortly.