Rima Das
[ 00:00:46,250 ]Okay, we might kick things off. Welcome, everyone. Thanks so much for joining us today. My name is Rima Das. I'm the Workforce Development Director here at RMIT Online.
Rima Das
[ 00:00:55,130 ] I'd like to start by acknowledging the people of the Woi Wurrung and Boon Wurrung language groups of the Eastern Kulin Nation on whose unceded lands we conduct the business of the university. RMIT respectfully acknowledges their ancestors and elders past and present. RMIT also acknowledges the traditional custodians and their ancestors. of the lands and waterways across Australia where we conduct our business.
Rima Das
[ 00:01:18,890 ] Before we get started, some quick housekeeping.
Rima Das
[ 00:01:21,640 ] Please feel free to drop your questions into the chat throughout the session today. We'll be addressing them all together at the end. When you do post a question, make sure that you send it to everyone so it is visible to all panelists and attendees.
Rima Das
[ 00:01:33,310 ] Today's session is in partnership with Deloitte Access Economics. The session is about going beyond prompting to what actually matters in practice. You'll have the chance to reflect on your own AI habits, identify common capability gaps, and explore what it takes to move from more basic use to effective AI collaboration.
Rima Das
[ 00:01:51,890 ] If you haven't had the chance to read the report just yet, that's okay. We'll be sharing a link in the chat towards the end. So please make sure you grab your copy. I'm joined today by Rhiannon Yitzinger, Associate Director at Deloitte Access Economics. Welcome, Rhiannon. And Jonathan McCormick, Partner Consulting at Deloitte. Welcome, Jonathan. I'll now hand over to Rhiannon and Jonathan to get us started.
Rhiannon Yetsenga
[ 00:02:14,740 ] Awesome. Thanks so much, Reema. And so great to be here today and have the opportunity to do a masterclass with you all. Um, I'll just make sure my slides are working. Fabulous. Okay, so today's masterclass is based on a piece of research we released recently with RMIT Online called 'Beyond Prompting, Measuring the Generational AI Gap.'
Rhiannon Yetsenga
[ 00:02:38,820 ] And at Deloitte Access Economics, we do a piece of research each year with RMIT Online, generally focused on kind of digital skills and the workforce. And when we were thinking about, you know, what... topic to do this year. We obviously couldn't go past AI. I think every second report I do at the moment is talking about AI. And there's a reason for that. It's really important. It's really disruptive. And it's a really, I think, exciting time for everyone.
Rhiannon Yetsenga
[ 00:03:05,130 ] When we were thinking about how to kind of frame this research, you know, we didn't want to add to the noise on AI. There's already a lot of research that exists and we're very, very cognizant of that. And so there were kind of two research questions that we... sought out to answer when we're approaching this work.
Rhiannon Yetsenga
[ 00:03:24,790 ] The first was, what does being AI literate really mean? Like, if I put 'skilled in AI' on my resume, what does that mean? Because anyone can use ChatGPT, anyone can use Copilot. So we really wanted to understand the difference between using AI and using it well. And then the second research question we had was, well, where are the gaps in the workforce? And specifically, we wanted to approach this from a cohort lens. And then even deeper than that, we wanted to approach it from a generational perspective. So, how are different generations using AI, and how does that differ? Our hypothesis, of course, was that younger generations, digital natives, might be, you know, better at using AI. That's certainly what we see with other digital skills, but we wondered if there was a more nuanced story when it came to how different generations are engaging with AI.
Rhiannon Yetsenga
[ 00:04:21,040 ] To inform this research, and just when you see, when we share a few stats on the screen, just to give you a sense of the data, we did a survey of about 2,000 workers across the Australian labour market, broadly representative in terms of things like gender and age and position type and sector, so we can kind of extrapolate the findings.
Rhiannon Yetsenga
[ 00:04:38,950 ] We did a literature review and then we drew on some other publicly available data, for example, from the ABS.
Rhiannon Yetsenga
[ 00:04:45,350 ] Hopefully you guys get a few things from this session. You'll see them on screen, you know, recognize what using AI well means, you know, learn how to use it effectively, identify common gaps, understands the AI beginner tax, so what it costs you if you're actually not using AI well. We did some funky modeling in that. I'm an economist, so I'm really... you know, a bit of a data nerd and love modeling. And then also just provide some insights into building AI confidence and capability.
Rhiannon Yetsenga
[ 00:05:12,990 ] One flag before I get stuck into the findings that I just wanted to say up front is this is not a how-to prompting session. So we do have some practical suggestions for you and hopefully that will give you, you know, insights into how you can better use AI, but it's not a how-to session. What we really want to do here is elevate the conversation. We want to get you thinking about how you can use AI more strategically and more holistically, and really think about what drives the most value and where the gaps are. So we don't think there's a one-size-fits-all approach. So hopefully, you'll see this more nuanced narrative coming through the presentation today.
Rhiannon Yetsenga
[ 00:05:53,010 ] Oh, final thing before I get stuck into the findings. This is the first of two reports. We have a second report coming out mid-year, which is all about trust in AI, attitudes, and well-being. So please do keep an eye out for that if you're interested in this space.
Rhiannon Yetsenga
[ 00:06:06,050 ] Okay, I'll get started.
Rhiannon Yetsenga
[ 00:06:09,360 ] So AI use in Australia is quite widespread. According to our survey, we had about 84% of workers who are using at least one form of AI in their jobs.
Rhiannon Yetsenga
[ 00:06:19,800 ] Yet, just 7% of the Australian workforce has advanced literacy and more than half have beginner AI literacy. And when we extrapolate that out across the labour market, that implies more than 5 million people are just beginner users of AI.
Rhiannon Yetsenga
[ 00:06:36,690 ] So what this means is that the adoption of AI is racing ahead of our understanding of it. Many people are using AI, far fewer people are using it well.
Rhiannon Yetsenga
[ 00:06:48,520 ] For our research, we developed a new kind of AI literacy framework, and it ranked people on a five-point scale from beginner through to advanced across six domains and two broad kinds of categories of skills.
Rhiannon Yetsenga
[ 00:07:02,680 ] So the first broad category was technical skills. You'll see them on the left-hand side of the screen in that purple color. And that's really just thinking—do you know, you know, do you know what AI tools exist? Are you using them? Can you prompt well?—really just the basics behind using AI.
Rhiannon Yetsenga
[ 00:07:18,960 ] But what we also added to this AI literacy framework was what we call judgment skills. These are in yellow on the right-hand side of the screen. So these are things like: are you critically- evaluating AI outputs? Are you thinking strategically about how and when to use AI in different contexts? And are you thinking about the ethical and legal risks? Of using AI in different contexts. Now, most existing frameworks really focus on the technical side of things, but we thought it was really imperative to actually focus on those complementary judgment skills because we think that is both where the risks are, but then also where the opportunities lie when it comes to using AI.
Rhiannon Yetsenga
[ 00:07:57,910 ] So I might just ask, Johnny, do you have a comment or do you want to provide some context to the group around, you know, you work with teams at various levels of this capability. Yeah. Can you elaborate on what you're seeing?
Jonathan McCormick
[ 00:08:08,320 ] Yes, if you want to think about it and put some faces to the framework of beginner and an intermediate 3D advanced user. Um, I'll give you a couple of examples and explain them because the minute you do that, leaders stop asking, 'Are my people using AI?' And they start asking, 'Well, um, are they using it well? And what does actually good look like?' So to take the beginner— Um, they use AI like a better search engine. They piece the question in, they take the first answer and they move on. They probably have no clue what model it is and they have no idea whether it's right or it's wrong. So that's kind of the beginner level. The intermediate user uses AI like a drafting partner.
Jonathan McCormick
[ 00:08:50,410 ] So they'll iterate, they'll refine it, they'll go back and forth. They'll catch obvious mistakes that the AI will spit out. And maybe they're the kind of person who has favorite prompts. They've got a little text file on their desktop and they've got all these kinds of prompts they can copy and paste in the use. And they're genuinely saving real time.
Jonathan McCormick
[ 00:09:09,360 ] So that's the intermediate. So advanced then will... They use AI like a thinking partner.
Jonathan McCormick
[ 00:09:15,890 ] Um, they know when not to use it, all right. They can spot a hallucination a mile away. And so that kind of person that might build small workflows and then they'll share them with their colleagues.
Jonathan McCormick
[ 00:09:40,230 ] Where we see organizations, we work with them, they sort of tap out at the intermediate user level in general. But moving from intermediate to advanced that's where the real productivity is and that's where beyond prompting actually starts. Prompting and good prompting and prompt like a boss and all those kinds of courses get you to either intermediate. It's that judgment, that right hand side. That's going to get you to Uh, advanced, um, and so I'll give you a couple of industry stats. And these are all publicly available things. These aren't from clients in that sense.
Jonathan McCormick
[ 00:10:15,280 ] Atlassian's internal usage data shows that the gap between the top usage and the bottom usage of AI users, even in the same team, is often 5 to 10x. So, same rule. Same tools. But, but just certainly that's a massive difference. And so. Um, I would look at this and I would absolutely agree with this is a great framework. The one thing I would stick my neck out and say is there's probably a seventh dimension here. And it's psychological safety. And where we see organizations that, or teams, even, and just zoom inside the organization.
Jonathan McCormick
[ 00:10:49,530 ] Where, where. They're in a team where they are quite open to say, 'I used AI to produce this,' so they're not going to get judged. They're not going to have people go, oh, what are you playing at? You should do it all yourself. That's the sort of indicative in terms of getting from an intermediate to advanced. Whereas the sort of beginners are most always in teams where it's not acceptable to talk about. As an example of where AI is being used. So hopefully that's a helpful insight.
Rhiannon Yetsenga
[ 00:11:18,220 ] That's awesome. Thank you. Yeah, so I think that empowering point is really interesting as well. You know, do you feel comfortable and confident in your organization to actually use AI and experiment with it? Really critical part of use.
Rhiannon Yetsenga
[ 00:11:31,950 ] So Johnny's just spoken about kind of differences between the different capability levels. I want to tell you a little bit about what we saw amongst workers. And we kind of applied this framework to the labour market and where we actually saw the bigger gaps. So what we found is that workers... are twice as likely to be advanced in technical AI skills compared to judgment skills, where only 11% of people are advanced. So what this really implies is that people are learning how to use the tools faster than they are learning how to evaluate the output. And we think that that creates a real risk because scaling AI use without that critical judgment means basically scaling AI mistakes.
Rhiannon Yetsenga
[ 00:12:19,989 ] I also don't think— and I don't think anyone's core would be particularly surprised to see that result. You know, I think technical skills are a lot easier to teach. easier to practice, kind of easier to measure. Judgment skills probably takes longer, requires a little bit more experience, requires feedback, you know, that willingness to be wrong. All of those things are part of building those judgment capabilities. But we think the judgment capabilities are really what sets apart whether AI is actually productive or a detractor or actually quite damaging in some circumstances.
Rhiannon Yetsenga
[ 00:12:57,090 ] So I want to show you a little bit now about how this is playing out across different generations, because the story is not as simple as, you know, younger people are better at AI. So let's jump to that.
Rhiannon Yetsenga
[ 00:13:10,710 ] So when we considered this from a generational perspective, yes, we found younger generations generally more skilled at AI. This mirrors what we're seeing across other digital skills. And of course, you know, younger people, generally digital natives, more likely to experiment, a bit less cautious about trying something new. I just want to flag when I'm talking about generations is that we know generations are not homogenous. There are people much older than me who are much more skilled than me, and people much younger than me who are much less skilled. So we think the generational angle gives us kind of useful averages, but we're completely aware of the limitations of this and just want to recognise there are differences within generations.
Rhiannon Yetsenga
[ 00:13:52,840 ] So younger people, yes, higher AI literacy on average, but is there a more nuanced story?
Rhiannon Yetsenga
[ 00:14:00,100 ] When we actually looked at younger generations in particular, what we found is that younger generations are much more likely to be overconfident. You can see that on the chart on the left-hand side of the screen.
Rhiannon Yetsenga
[ 00:14:11,980 ] About 17% of Gen Z are overconfident. And that compares to, you know, about 8% of baby boomers. And well, why does this matter? And it matters because... because overconfidence can create risks, either because you might misuse AI, but also because you might think you know more than you actually do, and you're not actually taking advantage of the full product. Dividends that come from using AI. We also know that the costs of using AI or misusing AI are not immaterial. In a recent survey, in a separate study that was done, they found that actually 95% of Australian businesses had reported experiencing an AI-related incident in the past two years, and that the cost of this averaged $800,000 per company. So it's very, very, very common and it's very, very, very costly. So for younger people, we kind of have this story of high literacy coupled with overconfidence, which creates, yeah, a bit of a risk.
Rhiannon Yetsenga
[ 00:15:10,130 ] With potentially damaging consequences.
Rhiannon Yetsenga
[ 00:15:13,250 ] Now, when we look at this from an older generation perspective, what we find is that they have lower literacy, but they're actually twice as likely not to have completed AI training. So the people with the lower literacy actually not upskilling in AI. And this, of course, reduces experimentation. It limits the productivity potential of the technology.
Rhiannon Yetsenga
[ 00:15:35,560 ] The other dynamic, which I think is really interesting when you're thinking about it from a generational perspective, is that older generations, just by virtue of kind of tenure and experience, etc., are more like... to be in decision-making roles. And in some cases, it means they're sort of setting the strategy for how organizations are rolling out AI within their organization or how they're investing in AI.
Rhiannon Yetsenga
[ 00:15:56,500 ] So, what that means is that you have kind of a cohort of people who aren't skilled, they're not training, and they're actually setting the strategy for their organizations.
Rhiannon Yetsenga
[ 00:16:08,370 ] I wondered, John, do you have a comment on just that leadership dynamic when it comes to AI upskilling and, you know, what happens when the situation kind of presents itself?
Jonathan McCormick
[ 00:16:20,100 ] Yeah, the generational data has an edge that doesn't always come through. Or on that, the ph— the most senior people in most organizations— are on average the least AI literate. Um, and if that's not a criticism— if that's the structural fact that the stats are showing— then And we do see that in organizations.
Jonathan McCormick
[ 00:16:41,680 ] They are the executive rules that hold the power around what are the types of training, defining the strategy. And so those people setting policy are often the least equipped to evaluate it. That is part of the challenge. And so that can play out in a couple of different ways. Either you get overcautious leaders who block, resist, and therefore create the shadow AI problem, which is people go off-piste, go outside the firewall, go use the publicly available tools and expose the organization's data to them. And that's what causes a lot of these issues that we see reported.
Jonathan McCormick
[ 00:17:23,910 ] Or equally, the other side of the coin is you get over-delegating leaders, you know, sort of the other cohort. We'll delegate to them, they'll figure it out. And so the risk is they abdicate the strategy entirely. So both are bad outcomes, both of those two extremes. But they both come from the same root cause.
Jonathan McCormick
[ 00:17:41,820 ] Leaders who don't feel personally fluent.
Jonathan McCormick
[ 00:17:45,570 ] And so there's a number of things— a number of proven things— to bridge that. Um, if I may, and and it's just start to think about you know, with Jack Welch's famous reverse mentoring that he did with his executives in the 90s. The reverse mentoring, with a good structure, really works. That's pairing a senior leader with a digitally native colleague. Not like a one-off coffee, but a sort of standing 30 minutes once a fortnight. With a real problem to solve. And, you know, the leader brings a problem and they work it together and they learn.
Jonathan McCormick
[ 00:18:17,500 ] I'm the beneficiary of that type of thing. So after about three months, the leader is competent. And after about six, they're an advocate. And we see that play out across industry.
Jonathan McCormick
[ 00:18:27,920 ] Another couple of things that you could bake into your process is we encourage people to show you're working as a sort of a ritual in a team meeting where, in leadership meetings, you ask one person to show how they did what they did using AI, and what have they learned, and what did they get wrong. And that's a really good way of normalizing the conversation. And what it actually also does secondarily is it structures the shadow experts in your team that you can go to.
Jonathan McCormick
[ 00:18:52,480 ] So there's lots of different ways to think about it. Um, I think that the most common sense response is: don't let leaders skip the training that they expect everybody else to do. So it's obvious.
Jonathan McCormick
[ 00:19:03,990 ] But it is the most violated rule that I see. You know, the CEO who exempts themselves from the training they've set for the whole company— they kind of set a tone, they sort of set that cultural ceiling— on AI literacy for everybody. So, we look at industry examples and we say, 'All right.' For example, Telstra's Microsoft Copilot rollout— they're public about was paired with leadership-specific enablement. Their public commentary has emphasized that leader behavior was the the single biggest predictor of team uptake, you know, so everybody's looking at you and, you know, thinking about your leadership shadow. That's it. It's a it's it's common sense in one sense, but it's just not practiced very commonly, as we would often joke. So there are clear examples in that. And the number to hold in your head for later is the report estimates about a three billion-dollar national wage value that sits inside that gap between the cohorts. and what we will probably call the boomer bonus. And that's not Gen Z productivity, that's a senior leadership opportunity.
Jonathan McCormick
[ 00:20:09,340 ] So. That leads to the question of how people are actually being trained today or, spoiler alert, how they're maybe not being trained. as well. So. If you take this slide and you look at the stats, Rhi has just shown us that adoption is racing ahead of literacy and that there's a sharp generational gap to where that gap sits. Now, a sharp shape to that gap. This slide tells us why.
Jonathan McCormick
[ 00:20:36,140 ] Less than half of Australian workers, 48%, have received any formal AI training from their employer. And then you look at the very bottom end there, only 11%.
Jonathan McCormick
[ 00:20:48,050 ] Get anything structured on an ongoing basis right? So you said that for a second: this is the biggest workplace tax shift in decades, and we're basically not training people.
Jonathan McCormick
[ 00:20:58,620 ] And when you dig into this with clients across industry, the picture is even starker than that headline. The training that does exist tends to be one of three sort of things, you know, there's the compliance webinar, 45 minutes. Uh, watched on 1. 5x speed and and mostly all the things that you shall not do—uh— and that's number one— and another one would be the really flashy bender demos—um— you know, people come in—Microsoft, Google, Open AI— come into a sparkling 60 minutes. People are razzle dazzled and then you know, afterwards they they go back to their emails. Uh, and then.
Jonathan McCormick
[ 00:21:33,820 ] There's other organisations where we build these beautifully large self-serve learning libraries with all these beautiful QRGs and great content, but the completion rates through that process is south of 15%.
Jonathan McCormick
[ 00:21:47,010 ] And none of those are bad.
Jonathan McCormick
[ 00:21:49,410 ] But they're all insufficient in their own. The pattern. That actually moves literacy is more, much more boring. Small cohorts of about eight to 12 people, real worked examples, and then follow-up sessions, and basically a manager who cares whether or not you've applied it. So Commonwealth Bank, they've been very public about its multi-year AI capability investment.
Jonathan McCormick
[ 00:22:14,990 ] A significant share of their investment has gone into people.
Jonathan McCormick
[ 00:22:18,870 ] Not just platforms. All right.
Jonathan McCormick
[ 00:22:21,560 ] The LEI Center of Excellence, they've got role-specific training, and so on. But the pattern is that organizations that are taking this seriously are spending on people, roughly at the same amount, the same scale that they spend on licenses.
Jonathan McCormick
[ 00:22:37,610 ] So on the next slide, we sort of look at, well, what fills the vacuum for training? And nearly half of workers, 49%, ourself. Guided learners.
Jonathan McCormick
[ 00:22:49,500 ] And what does that mean? Trial and error on the job, and if on the job, then there's really no feedback loop for them. Another one in five rely on peers or YouTube tutorials.
Jonathan McCormick
[ 00:23:00,300 ] Um, So roughly 70% of all the AI literacy in Australia today is being built by accident.
Jonathan McCormick
[ 00:23:09,180 ] Self-directed learning now isn't bad, don't get me wrong, it's actually how a lot of people get from intermediate to advanced. But the problem is, is this. What gets reinforced if there isn't a feedback loop? If there's no feedback in your environment, if you're prompt. produces a plausible sounding answer and you never really check it. Well, you'll go on producing plausible-sounding answers forever. And that becomes this kind of problem of confidence without competence, you know, and And so the fix isn't the barn. It's self-directed learning. It's to put a scaffold around it, and the kind of scaffold that I would suggest is things like: 'The moves that were worked are. Prompt libraries with bad examples.
Jonathan McCormick
[ 00:23:51,540 ] You've got curated beautiful prompt libraries of things to use, but what you should be pivoting to is having them tagged with bad examples and bad example outputs.' Not the what to do library, uh. The what not to do library. I should say is is more valuable than the what to do library because you're starting to lean in on the judgment side of things. You're starting to educate in that. You can have peer demo days, you know. Uh, really low stakes, informal, monthly, maybe. You know, uh, examples of, hey, I did this and this is how I did it in my day job and how it helped. And that is typically the single highest impact intervention when it comes to training. And then this whole concept of office hours, I've learned this term in the last month, office hours. It's kind of like a space or an online call where you've got a couple of experts ready and you can just simply come to them and go, 'Hey, jump on the call for 10 minutes and talk to them.' 'Hey, I've got a problem with this workflow, what's going on?' And they teach you how to fish and it's quite useful. So they're cheap, they're repeatable, they're peer-led, I think is the key.
Jonathan McCormick
[ 00:24:53,070 ] So you don't necessarily need it. A big, grand learning platform. Rather, you need a series of calendar invites. And that's the scaffold that kind of is there. So this is exactly what.
Jonathan McCormick
[ 00:25:04,230 ] Microsoft's Co-Pilot Champions Network. That's that pattern that they talk about. That's basically that scale. They deliberately seeded. peer advocates across the business units. To teach use cases laterally rather than top down.
Jonathan McCormick
[ 00:25:21,140 ] And so, on the next slide, well, this is kind of the punchline for this section. And if you're going to write a number on a post-it note, this is the number. Workers who receive employer support are 2. 6 times more likely to view AI skills as key to their next promotion.
Jonathan McCormick
[ 00:25:36,130 ] So let me unpack that a little bit because it's deeper than, hey, training is good. It's much deeper than that. Um, When an employer invests in your AI literacy, two things happen. One, you get more clarity. Okay, that's obvious. But two, you get a clear signal that this matters.
Jonathan McCormick
[ 00:25:52,750 ] That the organization expects AI fluency to be part of how it promotes people, how it evaluates and rewards.
Jonathan McCormick
[ 00:25:59,190 ] And that second signal is doing just as much work as the training itself. It matters.
Jonathan McCormick
[ 00:26:04,390 ] And when you change the game, people raise their game. And that's the kind of ethos that I'm talking about. And the mistake we see across clients and organizations is they invest in the training, but they don't change the signals around it. And they run a program, but yet they promote the same people and they go on their same way. And it's all the same criteria. And that would evaporate the 2. 6x And basically, you get this phrase, training without signaling is theater, which I guess is a beautiful phrase. Training without signaling. It's there and so the leadership question to take from this slide is: is brutally simple. Does your AI fluency show up anywhere in how your organization hires? or how it promotes or how it sets your goals. And if answers no, then you're going to pay for training that doesn't really compound. Uh, over time and that's exactly the lever that Reese is going to pull on next is: you know what, training that actually does compound. What does it look like in dollar terms?
Rhiannon Yetsenga
[ 00:27:02,750 ] Thank you. Yes, so Johnny's sort of outlined, I guess, the training ecosystem and really that it's a bit rare and self-directed and often unsupported by employers. And now I want to talk about why that matters. So we did some econometric modeling and what we found is that a one-point increase in AI literacy, which is roughly the equivalent of moving from beginner to intermediate literacy levels, is associated with a 6% increase in wages for full-time workers. And just to say that in another way, what it means is... That for the average full-time worker, moving from beginner to intermediate levels of literacy is associated with a $7,000 wage boost, and moving from beginner to advanced is associated with $11,000 wage boost. You can also reframe that into a beginner tax. If you have beginner levels of AI literacy, you're leaving potentially $11,000 on the table from not upskilling.
Rhiannon Yetsenga
[ 00:28:02,290 ] One thing I just want to note about the interpretation of these figures is that we control for a range of different factors. When we did the analysis, you can see some of them on the screen.
Rhiannon Yetsenga
[ 00:28:12,020 ] Within role finding. So this is for someone within the same role as someone else in the same occupation, the same industry. It's not from taking on additional responsibility. It's literally just from having higher levels of AI capability. So the beginner tax— or equally, the opportunity associated with upskilling in AI literacy— is certainly not immaterial for the individual employee.
Rhiannon Yetsenga
[ 00:28:37,890 ] We can also look at this from an economy-wide perspective. Of course, when you scale across the economy, you get much bigger figures. We looked at a situation where, if just half of all beginner-level workers upskilled to intermediate level, and the aggregate wage dividend is just shy of $19 billion.
Rhiannon Yetsenga
[ 00:28:58,660 ] So this isn't about, you know, everyone becoming an AI expert, you know, everyone becoming advanced capability. This is just half of all beginners upskilled to intermediate level. This is the wage opportunity or the benefit that exists.
Rhiannon Yetsenga
[ 00:29:13,240 ] One thing I want to say about this figure is that, specifically here, we're talking about individual-level wages, the wage boost. There are much broader benefits that come from using AI well. It's a significant driver of productivity. It has significant impacts for innovation. So the actual benefits in dollar terms could be much, much bigger than this $19 billion figure, which is already pretty substantive.
Rhiannon Yetsenga
[ 00:29:40,890 ] Johnny kind of spoke to this earlier as well, but we can also look at this from a cohort perspective. And specifically, we were interested in understanding what the benefit is to boomers. So what we found is, if the average boomer, the average level of literacy amongst boomers, was the same as the average level of literacy amongst millennials, the payoff is... about $3 billion.
Rhiannon Yetsenga
[ 00:30:02,540 ] We've spoken so far in the conversation that, you know, older generations are more likely to be in senior roles. Therefore, upskilling, you know, these senior decision makers can have, you know, a range of fallen benefits, unlock better decision making.
Rhiannon Yetsenga
[ 00:30:18,630 ] Ensure better policies, lead to better investment in AI across the whole organisation.
Rhiannon Yetsenga
[ 00:30:23,929 ] So again, like the $3 billion is, I would say, on the conservative side of things, but it also just demonstrates.
Rhiannon Yetsenga
[ 00:30:31,490 ] Yeah, the potential benefits that exist.
Rhiannon Yetsenga
[ 00:30:35,470 ] Just one more thing on the productivity side of things before I move on.
Rhiannon Yetsenga
[ 00:30:41,400 ] There is also an enormous time savings that comes with using AI. And I think this is often kind of the metric that most people think about when they think about how... How AI Drives Productivity. So in our survey, what we found is that on average, workers report saving about nine hours a week from using AI. Hey, that's not, that's... not a small number, you know, that's more than one day a week that comes from just using AI.
Rhiannon Yetsenga
[ 00:31:06,770 ] Yes, quite a powerful time savings. But what we found is that those savings were quite uneven. So you can see that on the slide. Younger workers saved the most time compared to older workers. The interesting aspect is that what we found is that younger workers saved the most time, but they were also the people more likely to feel pressure to work faster. And I think this is something that organizations really need to look to manage is that they're more literate workers— potentially just use that time to kind of reinvest in doing more work, which can lead to kind of risks from a well-being and burnout perspective. So. A few big numbers about the productivity story, and also something to watch out for. The next part of the conversation, which Johnny will lead, we just wanted to tell you a little bit more. The practical question— well, what do you actually do about it? So, what's the best way to upskill in AI?
Jonathan McCormick
[ 00:32:09,380 ] Thanks.
Jonathan McCormick
[ 00:32:10,730 ] So, if there are two skills you should be writing in your AI training program tomorrow, this slide names them based on the report and the stats.
Jonathan McCormick
[ 00:32:20,860 ] Transferability and critical evaluation. So, transferability is just reverse order. It's applying it fluently across different tools. and different contexts, so I'll unpack that a little bit more. Critical evaluation is being able to spot hallucinations and understanding the bias. And the risk aspects. And they're the biggest gaps across every generation in the survey data. But they're also the hardest to build by accident in terms of that self-directed learning. Um And that's why these matter more than just simply, 'hey, here's how to structure a better prompt.' Because that's what most training obsesses over— the quintessential prompt. I would see transferability as the antidote to, I only know how to use Copilot as a kind of a... Phrase of our position and if your literacy is locked to one tool every time there's a model upgrade, the latest and graded 5. 4, 5. 5, and 5. 6, and all the new razzmatazz that comes out, or every new vendor, every product change. Well, what it does is it resets your competence.
Jonathan McCormick
[ 00:33:24,220 ] Um, Transferable skills mean that you can land on any new tool and be productive in about 20 minutes. That would be a strategic capability for your workforce, not just a feature for a product and a knowledge in that sense. Critical evaluation is the skill that prevents the $800,000 incident that the report mentions that we mentioned earlier. It's the difference between an AI-generated draft that you trust and one that you check.
Jonathan McCormick
[ 00:33:49,400 ] And importantly, it's the skill that's least well taught by self-directed learning. You know, by definition, you don't know what you don't know. And so you don't know what you missed.
Jonathan McCormick
[ 00:33:59,370 ] So what does actually training look like for these? A couple of practical training moves for transferability would be rotating people through two or three tools deliberately.
Jonathan McCormick
[ 00:34:10,969 ] Run the same task on CodePilot, run it on Gemini, run it on Cloud. Compare the outputs, discuss why they differ.
Jonathan McCormick
[ 00:34:19,330 ] Suddenly, your people stop becoming co-pilot users and start being AI literate workers who understand what a context window is, the model's short-term memory. What models are better for design, coding, or knowledge work? And why, and their different flavors, because that's quite useful. For critical evaluation, you need to deliberately, back to my earlier point, seed known bad outputs into your training material. Run on the spot. Spot the hallucination. Workshops is kind of a way I would frame it. Build a bank of real... Your misses from your own organization and teach from them and it's kind of like analogous to you know those phishing emails that we get like the organization from a cyber security sends you a test and you kind of have to spot um well there's a genuine value in that as an analogy for the critical evaluation around models And the sort of tagline that I would leave for this slide is train the judgment, not the keystrokes. Train the judgment, not just where to click, okay?
Jonathan McCormick
[ 00:35:21,600 ] Microsoft have a publicly available responsible AI standard. Deloitte published the trustworthy AI frame. Look, it's sort of clean. examples of building critical evaluation into how your workforce is expected to operate, explicit expectations for things like verification.
Jonathan McCormick
[ 00:35:40,910 ] contestability and human oversight.
Jonathan McCormick
[ 00:35:43,660 ] It's better that that's how you train, rather than policy sitting on the shelf, all right? Goldman Sachs, quite famous and quite mature, they rolled out their internal AI assistant. To thousands and thousands of staff, with verification rituals actually built into the user experience, the UI.
Jonathan McCormick
[ 00:36:00,320 ] Every output has to have a human check and every check gets logged. And then they can learn from the the logged database so they build that muscle into the workflow as well.
Jonathan McCormick
[ 00:36:11,319 ] And so on the next slide, then we sort of want to look at if, you know, if you ask employees what do they want, they'll tell you. And the survey did exactly that. So they want two things.
Jonathan McCormick
[ 00:36:20,700 ] In this order. Number one, concrete job-specific use cases.
Jonathan McCormick
[ 00:36:25,700 ] And secondly, clear rules. But it was permitted. In other words, you know, the smooth the path kind of concept.
Jonathan McCormick
[ 00:36:34,960 ] What they're not looking for is a TED talk on artificial general intelligence and where we're heading. They don't really care for your vision document.
Jonathan McCormick
[ 00:36:42,620 ] With the greatest respect. They want to show me what good looks like in my role. All right, just tell me, tell me what I'm allowed to do with my company data and keep it really simple. And I think that's, in my experience, the biggest, I guess the most underserved aspect in corporate AI rollouts.
Jonathan McCormick
[ 00:37:00,860 ] You know, organizations love big abstract statements of principle and employees want a 90-second video on, well, this is how a sales rep at my company did this piece that prepared them for meeting faster and better. And here's the bit where they did the verification: they checked the output, you know, important to keep that crease in. So the kind of practical moves that I might suggest is build a use case library by role. With worked examples, not a hundred use cases, like 10 really, really killer use cases with prompts. With inputs and outputs. Um... and the failure modes explicitly within that, and that would probably be the highest return of investment training artifact that you could produce. And it seldom appears, but the stats obviously. Here reinforced on And the second thing, tongue in cheek, write a one-page AI policy that fits in a fridge.
Jonathan McCormick
[ 00:37:50,310 ] Most AI policies are 14 pages of legal hedging.
Jonathan McCormick
[ 00:37:54,600 ] Nobody reads them.
Jonathan McCormick
[ 00:37:56,430 ] Good ones answer five questions. Which tools? Which data?
Jonathan McCormick
[ 00:38:01,770 ] Which decisions need a human? Who do I ask and what do I do if something goes wrong? Both things. And if your policy can't answer those in plain English, what will happen is people will just default to paralysis.
Jonathan McCormick
[ 00:38:13,700 ] Um or or shadow it, they'll just go outside and use the tools that are available for free.
Jonathan McCormick
[ 00:38:19,180 ] Um, let's look at an example, Telstra. publicly discussed this approach, they've emphasized exactly this, clear permitted use case guidance, paired with role-specific enablement, all right. So there's the use case specifics tied into your rule. Rather than this big, corporate webinar kind of approach. Um, and Atlassian is pretty good in the sense that they publish much of their internal use case library externally. Partly it's good marketing for them, let's be frank, but it's also a great model. And they started narrow on purpose and they expanded that library as Confluence grew.
Jonathan McCormick
[ 00:38:57,050 ] So then, moving on to the next slide, here's where it all comes together.
Jonathan McCormick
[ 00:39:02,350 ] So AI upskilling shouldn't just be a single program that everybody gets through identically. It needs to bend to the workforce you actually have. And so you've got two different programs, maybe for two different problems. So for that younger cohort, on the left-hand side, what did the survey say? Well, the risk is overconfidence and misuse. Their top survey question was clear rules. So you have a program that leans into you. Ethics, judgment, and verification— that really important piece. The skills that they are least likely to build by trial and error, quite frankly.
Jonathan McCormick
[ 00:39:39,279 ] Guardrails and approved tools will smooth the path, as it were, and will turn enthusiasm into safe productivity. I know exactly what I can do. More data and get off to the races, and of course, visible recognition for good AI use is really quite important. Making judgment a status signal to the organization rather than a compliance checkbox. That somebody caught something, that somebody applied judgment. They were critically evaluating what the output was. They were starting to use it as a drafting partner and a thinking partner. Rather than just copy-paste, buying there's my answer and it's perfectly okay. We all can tell. The conversations I have with clients is I get so much AI slop from people. They don't write like that. It's so obvious. I mean, that's a conversation to have in your teams and in your training.
Jonathan McCormick
[ 00:40:28,090 ] And then the other cohort on the right, the risk— if it's hesitancy, if it's under use of tooling— their top ask was concrete use cases. So your program needs to lean into structured, hands-on small cohort training with peers, not the big glorified self-serve library. Approach they want—job-specific use cases that that obviously apply to their day-to-day work, not these beautiful shiny abstract demos. And in all cases, visible leadership endorsement, peer-to-peer learning matters disproportionately.
Jonathan McCormick
[ 00:41:03,290 ] At the senior levels.
Jonathan McCormick
[ 00:41:06,250 ] Ray's already alluded to this, like, 'don't... Fall into the trap, don't stereotype.' You know, let's be fair— the stats show that variation within generations is bigger than the variation between them. Um, some of the best AI users I work with are my age and above. Um, you can guess some of the worst are in their 20s. It shows the the the excitement and the enthusiasm, these good guardrails and training and enablement in that context.
Jonathan McCormick
[ 00:41:36,790 ] Use the sort of generational lens just to define defaults. Okay, like let's not be too explicit about it but let people opt up or down based on where they actually sit on the framework.
Jonathan McCormick
[ 00:41:46,220 ] And so you can have a little bit of a 10-question diagnostic with people up front. A little bit of self-assessment against the literacy framework and then route them into a couple of pathways for them. Uh, you know, into your content, so you can have the same content library and just different entry points. And that's how you tailor at scale without this massive curriculum nightmare.
Jonathan McCormick
[ 00:42:08,560 ] So just to round that off, NAB has spoken publicly about pairing AI training with their existing leadership development program. They recognize that the leadership cohort needs a different conversation.
Jonathan McCormick
[ 00:42:20,559 ] Bulling's Westmarmers they've emphasized use case training tailored to actual work. In other words, frontline retail demands a different training set from head office, okay?
Jonathan McCormick
[ 00:42:31,500 ] If I had to leave you with three words from these last three slides. Judgment.
Jonathan McCormick
[ 00:42:37,150 ] Use cases. and tailoring. Train for judgment, not the keystrokes, not the clicks. Lead with use cases, not your principal statements. And tailor the program to the workforce that you actually have.
Jonathan McCormick
[ 00:42:50,340 ] So hopefully that's useful soundbites to anchor to read back to you for Q &A.
Rhiannon Yetsenga
[ 00:42:56,180 ] Awesome. Thank you so much. I hope you've all enjoyed the presentation and got something from it. You know, a few summaries from me, you know, lots of Australians are using AI, far fewer. Using it well. Employer-supported training, really, really, really critical. Hey, $11,000 dividend if you upskill from beginner to advanced and clear use. Cases and really solid guard rail is really important from a training perspective. I've also seen a bunch of questions in the chat about getting access to the deck. Yes, happy to. share it. We also have a whole report which you can read, which I've been told is very long, but for those very enthusiastic, it has a lot of different sub-points we didn't get to cover in these 45 minutes that you might be interested in. Reema, I'm not sure if we have time for questions.
Rima Das
[ 00:43:42,249 ] I think we might. Yeah, Rhi, that was great. And thank you both so much. Thank you, Johnny, as well. I feel like we could discuss this report all day. There's so many valuable insights. We will share a link. We might just go over time a little bit if that's okay. Stay back five minutes so we can take some questions. Before I dive into the specific questions, I will say... It's been quite a few questions that have come through regarding specific courses. So please reach out to us at RMIT online via our website for individuals or via LinkedIn for Workforce Solutions. We will also put an email address in the chat so that you can ask your specific questions of the right channel there.
Rima Das
[ 00:44:17,610 ] But thank you so much to everyone that has put questions in. We've sort of picked out first in best dress style for this. So I might just quickly and happily for either. Johnny, for you to self-select, okay, all right, great, fantastic, that makes life easier. Um, so the first question is: do you have any research data in the Australian market to see how many people are concerned about their employability? For example, how many people think AI is going to replace them?
Rhiannon Yetsenga
[ 00:44:45,080 ] Yes, I can take this one. So a couple of points that I wanted to say on this. The first is that our next report covers AI anxieties. It's actually in the title. So please, that will have a specific stat which speaks to that specific question. So shameless plug for the next report. I think it's fair to say, though.
Rhiannon Yetsenga
[ 00:45:06,750 ] As a bit of a spoiler, there is a level of concern about the potential impact of AI on replacing people's jobs. And that's fair. The latest data from the Productivity Commissioner, which was reported recently, was that AI would replace about 4% of jobs and augment about 30%. It's very possible that both of those two things are actually much higher. I would say that's on the conservative side.
Rhiannon Yetsenga
[ 00:45:35,110 ] What we have seen from previous waves of digital innovation, however, just a cautioning point, is that technology does not replace jobs, it augments them. And I think... that that's, you know, I think we'll see what happens over the next few years. The thing with AI is it is changing and evolving so rapidly, including for example with the evolution of agentic AI, which is obviously the big thing at the moment. So it's hard to say with certainty what will happen. There is an enormous opportunity that comes from using AI really well. And what that means for our productivity. So I think I'm personally, I'm probably a bit more, you know, a bit more positive about the potential of AI and what that means. I don't think it's going to lead to mass wide scale, you know, unemployment. It certainly is a disruptor. We've already seen that and we are seeing that in some organizations, but how it plays out over time will be in part related to how well we're actually able to use AI to augment our positions and build our AI literacy to the reports point.
Rima Das
[ 00:46:36,420 ] Another question that sort of builds on that is: we have a very risk-averse culture around data and tech. So what is a company that's a standout for getting that right from your experience?
Jonathan McCormick
[ 00:46:47,880 ] It's a fair point in terms of the appetite and risk.
Jonathan McCormick
[ 00:46:53,539 ] The standouts aren't just a well-engaged set of leaders who know exactly where they're going and what they want to do. And from a top-down point of view, it's the enablement where they've invested well in their data.
Jonathan McCormick
[ 00:47:07,340 ] Rubbish in, rubbish out is still a very important aspect. And you can still apply AI to bad data and get it cleaned up quite quickly, which is an interesting paradox.
Jonathan McCormick
[ 00:47:18,050 ] Certainly those that have worked through their governance know what they have. And it's the basics of the last 10 years from a data engineering perspective that some companies may have ignored and just played lip service to. And so those that have good foundations, those that have good owners, can actually hit it at speed. And then, of course, as I led into it, the leadership component and where they want to go, and that psychological safety of the encouragement for people to get stuck in is something that I've really seen work. And that is all within a well thought through set of guardrails, that kind of concept of smoothing the path. If people know um that the governance is there to allow and enable not just to say no then that that is a massive accelerator too so there's a number of different dimensions that are in there but invest well in your your foundations invest in your your leadership signals in the organization and then the training that we talked about isn't just hey tool training it's making sure that you're having a genuine conversation about the sort of judgment and the critical enablement and not It's just simply sweating the workforce asset and making them do more.
Jonathan McCormick
[ 00:48:24,570 ] But understanding what does it actually does for the job and what does it actually does to restructure the role expectations to get people a bit more headspace, to be a bit more creative in that concept of having more time and having the kind of digital worker with them as well. Um, I'll stop there. I could go on, but those are sort of the key headlines for me.
Rima Das
[ 00:48:45,210 ] Yeah, great. Thank you, Johnny. There's one more that sort of piggybacks off something that you mentioned. So probably last two questions. This will be the second last. This one sort of, you spoke about leaders self. sort of withdrawing from certain training programs earlier when you were talking, when you were sharing your insights. And this question is aligned to that, but probably want to hear. Of the people side of things in terms of the, you know, frontline or middle management as opposed to the exec. So the question is, is the lack of AI training coming from organisations not preparing individuals, or from people absenting themselves from that training?
Rhiannon Yetsenga
[ 00:49:28,009 ] I just have an initial comment on this, which is that when we think about AI and how that's sort of played out in the market, it is quite different to previous. Waves of digital innovation because it has been consumer-led. So typically when we think about disruptions of digital technologies, it's actually enterprise-led. It's businesses saying, 'Hey, employees... please, please adopt this.' And usually that comes with a set of really specialized skills. You know, you need data engineering skills or what have you, really technical skills to actually use. leverage that technology effectively. The really interesting thing and quite unique thing about generative AI is that it was this consumer-led adoption. What we saw is that employees were actually adopting and using AI far before businesses even realized that was a thing, which of course led to a lot of risk. So I think this question of 'is it...' Is it, you know, is it the problem of the business or is it the problem of the people? I actually think that the training is just plain catch up. Businesses are just plain catch up with the wave of disruption that we've seen.
Rhiannon Yetsenga
[ 00:50:28,780 ] So that was just an initial comment. But Johnny, I don't know if you had anything to add.
Jonathan McCormick
[ 00:50:33,390 ] Oh, what I would add is relatively minor. The source suggests that while organisations are failing to provide the structured, job-specific training, the stats show that, that whole 50%— comment employees the the What they're opting out of training in the different cohorts is either overconfidence, to your point, perceived lack of value, too, you know, who cares? And the old sort of shouldn't use memes as a as a reference here but the old game on me you know people are um dragging a cart through of square wheels and somebody comes up with a circle and they go sorry too busy um is the kind of meme i have in my head for this one and it's like look people have a day job that they that their head down look in they're really uh busy and we're trying to then Not just give them tool training and they're going, oh, I don't really care, I'll pick it up. But what the conversation needs to be is to really amplify the value. It's like, all right, here's a new way of thinking about how you're doing and you're allowed to use this.
Jonathan McCormick
[ 00:51:34,060 ] And so the perceived lack of value is, I don't really care. But also the overconfidence piece. And then that third bit is that I'm trying to allude to is: I don't really see. Um, how I'm allowed and people there is a dimension I i find you're talking to people it's like oh it either takes away from what's called the scientist persona a little bit of my ability, I can already do this and we need to say, 'Yup, we get that,' but this is going to make the boat go faster in a repeatable, structured way. So there's that kind of psychological safety piece I keep coming back to, which is giving people that freedom to experiment and that expectation almost. And as an organization where we try and do this to ourselves. So we've got about, in my cohort, about a thousand people. And we've gone through all the enablement and all the encouragement and all the rest of it. And you get to the point where you're... where we realized pretty quickly the expectations needed to change. to encourage, in that sense, as well to actually bake it into the performance framework to drive big populations of people towards, all right, they're taking this seriously, I need to look in, and that at least then gives them the headspace.
Jonathan McCormick
[ 00:52:39,420 ] To say well, this is important to me as an organization, I'm gonna, I'm gonna jump in.
Rima Das
[ 00:52:44,650 ] Yeah, wow. That's super comprehensive, Johnny. Sort of leading on from that, and I'm not sure if Brie, you want to jump into this one or we keep going, but I did promise the last question. And the last one is, what's your take on where the critical thinking and judgment skills should be developed before AI can be used well? and in a reasonable way. In the higher education sector, a lot of students seem to be using AI for their work without thinking much and without the support of AI, they find it difficult to problem solve or ideate. So... I have an example of an organisation that's actually turned off a lot of its AI functionality and platforms until they upskill individuals and go through that training with us. So that's quite insightful, but we'd love to hear from both of you.
Rhiannon Yetsenga
[ 00:53:29,899 ] We didn't look at sequencing, but it's a really interesting question. What skills come first? Do you build the technical skills first or do you build the judgment skills first? I really like the question. I think my view is it's complementary because in order to be able to apply the judgment skills, you actually need to see the output and you need to know what you're dealing with. I think trying to explain that to someone in... in kind of a removed way would be really challenging. I noticed a question around, you know, how do we actually build these judgment skills in the context of AI? And, you know, I noted earlier that judgment skills are certainly... you know, on paper a lot harder to teach than technical skills. I think Johnny mentioned earlier that, you know, some better ways of actually training people in AI. are things like showing examples of bad prompts and bad outputs. I think that's a great way to teach judgment skills. This is not what not, like what we don't want you doing. And this is actually... Example of bad output. And this is the risk and the impact it had. So I think my view on sort of the sequencing is that it has to be complementary.
Rhiannon Yetsenga
[ 00:54:34,630 ] I also think the level of training, because we've seen across the market that different cohorts have different gaps. You know, for younger workers, we know that actually the judgment skills are really critical. And that's why you should focus your time. For older workers, maybe a little bit more experienced, you know, a little bit more easily able to have those judgment skills. Maybe for them, it's actually about the technical skills, what AI tools exist, etc. So I would almost say I think they're complementary, but I also think you should have a cohort approach and look at the strengths and weaknesses of the people you're trying to upskill.
Rima Das
[ 00:55:05,400 ] Yeah, I love that persona-based approach. Well, we've gone way over time today. I just want to remind everyone that you will find a link to download the report in the chat. So please look out for that. If you're interested in learning more about the space, please feel free to connect with any of us on LinkedIn. via Deloitte Access Economics and RMIT online as well. And thank you, Rhiannon and Jonathan, for taking time out of your day today to join us. We really appreciate having you here. And thank you to everyone who joined us today. We'll send out a copy of the recording as well as the slides. Have a great day. Thank you. Thank you. Bye.
