Frances Kleven: Welcome back to The Learning Signal Podcast. I’m your host, Frances Kleven. For this episode, I’m joined by Megan Torrance, Founder and CEO of TorranceLearning, and a leading voice in instructional design, agile project management, and learning analytics.

 

There’s a lot of noise around AI now, and learning leaders are under pressure to move faster while also taking a thoughtful approach. In this episode, Megan and I explore how to navigate that tension. From using AI to boost productivity without creating more busywork, to building leadership skills across flatter teams. We also look at L&D’s role as organizational first responders, and why it’s time to move beyond completion rates and focus on what really matters: performance. 

Let’s dive in!

 

Well, welcome everyone back to The Learning Signal Podcast. Megan, I’ve been looking forward to this conversation since we met two weeks ago, and that could have lasted for another two hours, so I feel like it’s a continuation of that. We geeked out on all things learning and development. This is very much in conversation with other interviews that we’ve had over the season around business transformation and personal productivity in the AI era. So excited to kick this off today.

 

Megan Torrance: Me too. And we did have a hard time finishing that last conversation. We might have a hard time finishing this one.

 

Frances Kleven: I know, we will. But there can always be a part two or a season two, so if we don’t get to everything, we can follow up with that. I wanted to start because I think it’s helpful. There’s so much noise in AI, and we were talking in a previous episode around how I think everyone feels just squished between the excitement and desire for velocity and speed with AI, and then also the “I’m not sure where to start” or “my organization is a little bit wary, so we’re squished between these two energies that are telling us to speed up and to slow down, and it feels like we’re a little bit stuck”. So I think framings are helpful here in just helping people categorize things or put vocabulary to things. I thought we’d start—I know you’re thinking about the zones of AI impact—and so I think it’s always helpful to start with ourselves. Talk a little bit about personal productivity and AI and how you’re thinking about that and how you’re talking to organizations about that.

 

Megan Torrance: Yeah, it’s a great place to start because it’s where we start. I think of these three zones of AI impact not necessarily as maturity or phases, but just three things that are happening at different times. I think, Frances, what you just said is really salient. We are not only learning about it and doing it and expanding our own skills, we’re at the same time helping everybody else in the organization do that too, right? So we’ve got two things going on, and that’s real. So in this zone one, it’s really all around our own personal productivity. How are we using AI? In some respects, in organizations we’re using AI similarly to everybody else, right? We have a conversational chatbot, and we’re using that to help us get our work done, and that’s great. But then we also have a layer of specialized tools for our industry that help us move faster, whether it’s embedded in the LMS or the LXP, whether it’s our authoring tools, or whether it’s our media tools—so all of our media editing and whatnot. We’ve got multiple sets of tools and capabilities that we’re all trying to get skilled up on at the same time.

 

Frances Kleven: And I think this is relevant to personal productivity, but also L&D metrics. One of the things we are always trying to measure in L&D is time saved—like whether our asynchronous onboarding program saved 10 hours of admin for in-person employee onboarding, or we were able to increase or decrease time spent in our engineering team by this training. This isn’t a new question, it’s something we’ve always been struggling with. But how do we wrap our heads around what we do with the time saved? Because I think there’s a real risk that we just fill it with more admin.

 

Megan Torrance: There absolutely is, right? Boston Consulting Group had a study out last month that shows basically we are filling it with other busywork. We’re also filling it with verification work, right? So I can offload some activities to AI tools, but I also need to check that those tools are doing it correctly. It’s kind of like moving from an individual contributor to a first-level leader, right? All of a sudden, somebody else is doing part of your old job, which is kind of great, but you’re still responsible for finding that person, coaching that person, reviewing that person’s work, and managing their performance. When they change and the environment changes, you have to manage that. So it’s not like magic. It feels like magic, but it’s not. We’re also doing more. I’m doing research with The Learning Guild right now, and we asked that question. It’s still preliminary, so I don’t have final results yet, but we asked what people are doing now that they are saving all this time with AI. There is absolutely more time spent in analysis. There’s more time spent in performance support. We’re not just cranking out courses. There’s more time spent in looking at the data and what’s going on. I know we’ll get into a conversation about data in a moment here if we follow the plan, which we are not necessarily guaranteed to do. But I think that there’s the opportunity for us to do some of the things that we have always felt are important in this job, if we get that opportunity.

 

Frances Kleven: And I think—and this has always been the case, this isn’t new—we have to remind ourselves that while it’s true we haven’t had time for some of those things, they’re also scarier. They sometimes are the thing where it is somewhat comforting to be able to say, “I don’t have time for analysis. I don’t have time for performance support. I don’t have time to reach out to my colleagues in other departments.” I think there’s truth to that, absolutely, but those are things that require a different type of energy and creativity. We have to acknowledge that so we can be intentional: “Okay, I am going to be spending my time in different ways, but it might be a little bit out of my comfort zone.” Or I might have to acknowledge that doing data analysis might tell me things that aren’t what I want to know—like that a training program isn’t as effective as I thought it was, now that I can ask immediate questions to get those answers.

 

Megan Torrance: Yeah. It’s always interesting, when people talk to me about data, they’re like, “I want to prove my value.” I say, “Let’s turn that question 90 degrees and find out what has value and how to improve.” Those are not always welcome statements on my part. I’m so much fun at parties. That can be hard.

 

Frances Kleven: I think that’s really interesting that you talked about leadership skills, because I hear a lot from organizations and companies that their L&D teams are thinking a lot about leadership development—not in the traditional sense of training managers, but in developing leadership skills for individuals. You are now becoming a manager even if you don’t have traditional direct reports, both in the way that you manage, get feedback, and coach yourself, but also because the access you have to data allows you to do manager- or leader-type functions. So for L&D, we need to develop those skills, but there is also a huge opportunity for us to offer that skill development internally to help individual contributors become leaders.

 

Megan Torrance: Yeah, especially in the framework of leading self, leading others, leading work, and leading leaders. This “leading work” looks different now because work looks different. Generally in this phase model or step model, they overlap a lot. Some of those critical thinking, analysis, and consultative skills are now becoming important to team members as part of what I just heard referred to as “the great flattening.” As organizations are becoming flatter, and as teams are becoming more cross-functional and expected to cover a wider swath, we have tools to do that. So I think that there’s a real thing there, and it requires that we learn and grow a lot ourselves.

 

Frances Kleven: I know that L&D can be so frustrating due to a lack of visibility in organizations and not always a lot of respect or understanding for what the profession does and the impact of it. But what a huge opportunity we have in L&D to say we could provide training that transforms organizations. In a different episode, somebody used the phrase “AI fluency”—if we could ensure that our entire organization is AI fluent and we delivered those trainings, that’s huge organizational change that’s delivered from L&D.

 

Megan Torrance: Yeah. The pandemic taught me that L&D is kind of like an organizational first responder, or maybe a first-and-a-half responder, right. When our organizations are in trouble or in times of great change, we have an incredibly powerful role to play, and we also have unique capabilities to do that. We have a great network within the organization—we know sponsors, new hires, and subject matter experts across the organization. We know more people in the organization, I think, than many other functions do, sharing that with our friends in human resources. We also, in many respects, understand the work because we’re teaching it, and we understand what it takes to do that. I like to think of L&D as the connection between process and people, and that’s how you get performance. That’s really what we’re very good at. I don’t know a single instructional designer who’s been on the job for any length of time who hasn’t run into a project where you look at a process and think, “Yeah, that’s not going to work.” We see this. That’s something of value we can offer. We can also facilitate conversations and bring people together, and we’re really good at that. So there’s real opportunity for us, and we’re getting better at measuring performance.

 

Frances Kleven: Yes, I think so. And I love that first responder analogy. During COVID, that was our moment to shine because it was so immediate and urgent: we had to retrain retail staff in a new way of working to keep stores open, or train formerly in-office staff on how to use remote tools so the business could keep running in a remote scenario. We were well-equipped to respond to that type of urgency, and I hope we can shift that a little bit here. There’s urgency, but in a different way, with so much opportunity.

 

Megan Torrance: Yes. Fortunately, it’s not life and death anymore. It’s not even the existential survival of the business anymore. But it is an opportunity for transformation and an opportunity to stay connected to the people involved. So there’s both.

 

Frances Kleven: Yes. Talk a little bit about the process side of things. How are you helping organizations and L&D teams think about process? We’re very good with processes and recognizing when something won’t work, but how do we need to think about our own L&D processes and what opportunities exist to level those up?

 

Megan Torrance: It’s interesting. I think some of the things that we as a function take as a given are changing. It’s interesting to see and at least for me it didn’t come instantly, with the arrival of ChatGPT in the public sphere. But I’m seeing a few things.  I’m seeing the opportunity to hyper-personalize learning. It used to be that you built a course for the primary persona who was going to attend, and everybody else came along even if you couldn’t meet everybody’s needs. It just didn’t make sense operationally or efficiency-wise to individually tutor every person. Now we actually have the opportunity to personalize delivery and coaching on a one-to-one level. In my book, there’s a story of a team that had a far-flung, small Eastern European office of four people who never got their own training. That meant they were always learning in a language that was not their native language—they had to take the German version or the US version and make their own modifications. The corporate training team was able to say, “No, actually, we can make it for just the four of you now.” That’s transformational. That hyper-personalization requires a mindset shift in learning design and development.

 

I think the other piece comes in the learning design process itself. Say, for example, I’m building an e-learning module. We have multiple checkpoints and review points along the way. When it’s in a Word document, PowerPoint, or Google Doc at that alpha draft storyboard spot, that’s where we ask subject matter experts to make all their changes because it’s cheap and easy there. We haven’t spent a lot of time or investment, and it’s easy to review. Then, when we build it out into that beta stage in the authoring tools and LMS, we tend to say, “What are just the little things you want to change?” because hopefully you caught all the big things when it was easy. Well, now that I can spin up the media side of it quickly, that changes. The problem with the old way of doing it is that when a course is in a Word or PowerPoint document, subject matter experts can’t really gel on what it is because they don’t see it in 3D. When it’s finally built out, it’s hard to change, and we get grouchy about it. Now that whole math changes, and I think that’s really powerful. We now have an opportunity to measure in that analysis stage, during development, and as we’re testing things, with more ability to analyze what’s going on to tune and tweak than we ever have.

 

Also, role-play tools give us the opportunity to have high-quality practice. I can’t tell you the number of times I’ve sat in a course and my role-play partner was not helpful. A role-play exercise is only as good as your partner, and if your partner is in class with you, they don’t know anything more than you do. It’s not set up well. So I’m actually very excited about role-play tools for situations where that’s a relevant construct.

 

Frances Kleven: Yes, it is, and we have to acknowledge the reality of what our in-person trainings were and where they weren’t always as effective as they could have been. I was talking to a customer who was describing exactly that: they do very intensive, high-stakes, expensive manager training with a lot of role play. He mentioned that for the first two or three role plays, people are nervous, cautious, and timid. Now that he knows there are other role-play tools and avatars out there, he realized that the first half-day of role play was not a good use of in-person time because everybody was scared and worried about looking silly or giving feedback. If you can do that in front of an avatar to get people a little more comfortable, when you put them together, you can actually accomplish something.

 

Megan Torrance: Yeah, absolutely. I think there’s a lot of power in that, and we can collect performance data and feedback from it. Centrally, we can figure out what’s going on, because as a facilitator in a class, you can’t have your eyes on everybody or find out what the common and unique challenges are. There’s lots of opportunity there.

 

Frances Kleven: Regarding hyper-personalization, we’ve always talked about personalization in L&D. But supervising customer education and being a people manager myself, I always knew personalization was important, but I ran the narrative in my head that I couldn’t do that at scale or didn’t have time for it. It’s an exciting shift, but not a simple one, to ask: what would a world look like where I can give personalized feedback to my team of Customer Success Managers at all times every day, rather than just a weekly one-on-one and a six-month performance review?

 

Megan Torrance: Yeah, I think there’s a lot of opportunity here and a lot of agency on the part of both the leader and the individual. As you were describing that, part of me thought, “Ooh, that’s a lot of feedback.” But if I had feedback every day, I bet I’d get so much better with that coaching. It’s interesting because professional athletes spend about 80% of their time practicing rather than playing in a game. It’s not just practice, but practice where they’re getting feedback. I play adult recreational hockey. There is no practice, no coach, no repetition, because everything is dynamic and different. When I say I’ve played hockey for 26 years, that sounds like I should be a lot better than I am! But I am slow, I miss the puck, I make bad decisions, and I fall down a lot. I do not look like I’ve been playing for 26 years because it’s been 26 years without consistent practice, coaching, or feedback.

 

Frances Kleven: The professional athlete analogy takes us to metrics and trust. In L&D, a lot of our relationship to or perception of learners—and our role—is that we need to monitor what they do. Did they do the thing? Did they pass the exam? Did they learn the material? That comes from traditional education where the teacher asks, “Did I take attendance? Did I grade the exam?” We’ve carried some of that into workplace learning. But with professional athletes, their coach is not grading them on whether they showed up for practice; they’re there to help them understand and coach their performance, with the understanding that the athlete will show up, be self-reflective, and look at data to improve. As we think about metrics differently, there’s a mindset change to establish trust at an organizational level that employees are going to engage in these activities, self-managing and self-leading. That’s transformative, but it’s a reminder for ourselves that our job is not just to measure completions. So how do we get out of the cycle of just measuring and reporting completions?

 

Megan Torrance: Measuring and reporting completions is a useful measure of activity—it’s not completely irrelevant, it’s just not the place where we stop. A lot of people are asking that question right now: “Did you complete the training?” In some cases, I may not even care that you completed the training; I want to know if you know what you need to know, if you can make the decisions you need to make in the right ways, and if you can perform the skill. I think of Will Thalheimer’s Learning Transfer Evaluation Model, and I’m making some tweaks around it. But starting with that model makes a lot of sense. There’s a lot of data collection around who came, what they did while they were there, and what they thought about that experience. That’s the most immediate and easiest stuff to get, and it has useful signals. But instead of just looking at who showed up and completed, I want to know what population segments did. It’s not just that we got 97% or 67%—who, why, when, and what made it effective for them? There’s more we can do with completion and perception data. Still within the learning experience, am I learning and practicing? What is the quality of that practice? Am I able to practice with less support or use performance supports? All of that is within our purview and data systems because we control that environment.

 

When you go out into the world for that transfer spot—whether it’s Level 3 or Kirkpatrick—that transfer spot is where it’s hard. Why? Because those outcomes are not measured in the systems the training team controls; they’re measured in the systems at work. In a worst-case scenario, they’re not measured, or they’re measured with a different index value, or at a team or shift level rather than an individual level, whereas we measure training individually. So there are data challenges there, and the average instructional designer isn’t in instructional design because they really like the math. Some of them are, and we have jobs for them! But there is a lot of really interesting possibility in that space.

 

Frances Kleven: It’s not as easy as snapping a finger to have an AI agent connect all our systems so an L&D leader can ask, “What’s the correlation between this manager training and the performance of our sales team?” You have to make sure those systems are connected, but once they are, you can ask that question and get an immediate answer, which doesn’t have to be a half-day’s work in a spreadsheet. We’ve never had access to that before.

 

Megan Torrance: Yeah, depending on the organization size and data structures, there are organizations who can answer that question right now, and others will get there. It’s a super exciting time.

 

Frances Kleven: It’s okay to start small. Asking your IT team for access to things can feel intimidating, and you know there are a lot of “no”s along the way. But you can start small: “Could we just connect these two datasets so I can ask this one question?” You don’t have to ask for access to all the data across the business right away.

 

Megan Torrance: We can also practice on our own data—the training side data—learn some things in that safe space, and demonstrate competence with it. Just before we started recording, I was working on a seven-page report for a client analyzing training data, and at the end I wrote, “And we look forward to connecting this to the operational data to show the real impact of what we did.” I look forward to that.

 

Frances Kleven: There’s a meta piece to this: it’s also our role to help the people we’re training understand how to use data as part of their learning experience. If you are on a sales team learning a selling framework, you now have access to your call recordings and emails to analyze that data. There is an opportunity and a responsibility for L&D to say, “Not only are we training you on this framework, but we’re training you on how to coach yourself through it and access data to inform the impact of your learning.”

 

Megan Torrance: That’s really insightful. It’s like having a mirror that I didn’t have before. If I could see that all the time, that’s valuable.

 

Frances Kleven: We probably have another three or four hours to talk, but we are coming up on time. I kept a note from our last conversation to end with a bit of wisdom for listeners. My question to you: how do we make the next rightest action the easiest and the most obvious?

 

Megan Torrance: I love that you borrowed that. The next rightest action is really the one that is going to help move the business forward. It might be sitting down with a business sponsor, a leader, or a subject matter expert in your organization and asking, “What would move the needle for you? What would be amazing for you?” L&D folks tend to be people-pleasers, and I don’t say that in a bad way—this is a service role. Getting outside of ourselves and asking the customer of our product, “What would be amazing?” lets us know what would be amazing and how we can fill that.

 

Frances Kleven: It’s true. When you said the “helper” piece, I had a slight reaction to that, but it’s important to reframe: we are here to help, and we are equipped to do so. We are equipped to hear from an engineering organization about their challenges in onboarding new engineers and their need to move faster, and say, “We can help with that.” Coming back to time saved, if you’re intentional about personal productivity and get an hour back in your week because an agent runs a report for you, use that time to speak with your colleagues and business partners. Really, don’t fill that freed-up time with spreadsheets.

 

Megan Torrance: I love it! That brings humanity back into the work.

 

Frances Kleven: Yes. Well, Megan, it was such a delight. I know we’ll have lots to talk about in future episodes or seasons, but thank you for sharing all of your insights, and we look forward to connecting on future projects.

 

Megan Torrance: Absolutely. This has been delightful.

 

Frances Kleven: Bye all!