How to Assess AI Readiness in Your Organization

00:00:06 Mark Smith
Welcome to the MVP show. My intention is that you listen to the stories of these MVP guests and are inspired to become an MVP and bring value to the world through your skills. If you have not checked it out. Ready. I do a YouTube series called how to become an MVP. The link is in the show notes. With that, let's get on with the show. Welcome back to the Microsoft Innovation Podcast. Today, our guest is from Copenhagen, and he's not only, you know, talking about AI. He's actually involved in implementing across organizations on a day by day basis. So welcome to the show Bo. He scales. No pilot across teams and designs AI that actually fits real use. This this conversation is packed with insights you won't want you or you won't actually find in a product demo. Full links are in the show notes for the episode. As always, both welcome to the show.

00:01:11 Bo Fischer Carlsen
Thanks a lot, mark and pleasure to be here.

00:01:14 Mark Smith
Good to have you on. You have a lovely country and I always like to start with getting to know you. So let's start with food, family and fun. What do they mean to you?

00:01:24 Bo Fischer Carlsen
Ohh so I'm a huge foodie and love both making food for myself and go out and eat. I mean with the yeah the next step which is the family with the two small kids, it's not that much anymore. But uh definitely cooking a lot in my home. Love Italian and South American cuisine so. A lot of doing a lot of. And then I have my wife and two sons on five years and five months. So yeah, busy home.

00:01:54 Mark Smith
Busy, busy home. That is so cool. So tell us about your specialty. Am I right in thinking copilot is is one of the key AI tools that you work with on a data to a day-to-day basis.

00:02:08 Bo Fischer Carlsen
Yeah, the the copilot is the core of what I'm doing. Uh, I come from a Microsoft 365 background or actually back to when it was SharePoint 27/20/10 and worked my way through Office 365, Microsoft 365 and that entire stack and now mainly focused on copilot for Microsoft 365. But also doing some agents doing a lot of other AI related stuff, all of it sort of fills together.

00:02:38 Mark Smith
How did how did you get involved in AI and and why AI at this time where you know prior to that I assume you're heavy in the document automation side of things, maybe Microsoft Teams. What's what's made you pivot and and go deeper into the AI space?

00:02:54 Bo Fischer Carlsen
So it's actually funny. I got hired into my current company delegate a couple of months prior to copilot being released by Microsoft. I was hired in to do document collaboration platforms. Contract uh platforms, doing security compliance stuff with the per view platform as well. Because I've done a lot of that. I've always tried to sort of bridge the security and compliance side of things with the employee experience, which of course they clash once in a while and trying to find that good. Bridge and then couple of months into that role took on a couple of the first customers then go Paul was there and I had a little spare time to start digging into that and found it sort of exciting that new revolution I had worked with Ted debt quite a while before that just more personal and sort of in my private productivity. See, so the pivot to Copal was quite easy, and of course it wasn't a dance in the start because Copal was quite immature. But I love finding those. So how can we actually use it at the stage it is now? And how can we see it evolve through the years on becoming more and more of a productivity? Cluster.

00:04:05 Mark Smith
Yeah, that's that's interesting. Cause you know, a lot of people would say there's been a lot of hype around AI, and I like your context there of right. What can I do with it in its current form? And so when you have those conversations with people, how how do you answer that? How do you answer 1? Is it just hype or is it a actual practical tool that we're all gonna end up, you know, making? It's going to be a critical tool that we use every day, kind of like, you know, Outlook is one of those tools that you can't really in the business world live without. An e-mail client right as an as As for communications and even now. Days in a more disconnected workforce, you would say Microsoft Teams with the video conferencing capability. That's probably also one of those indispensable tools. Everything else you could probably live without and still do business. But his copilot moving into that space.

00:05:02 Bo Fischer Carlsen
Yeah, I think more and more and I think there's still a lot of hype. There's still, of course, a lot of the Microsoft marketing going on about what can we do in 6-12 months. And I try to keep it on the side of what can we actually do now. And I feel it has pivoted over to something we can actually. Users day-to-day tool. It's still sort of my personal productivity, but it helps. But I had the sort of revelation a couple of months ago when copilot was down for half a day. I was at the point now where I didn't really realize how much I used it, but when it's gone, I realize how much I use it. Just with Outlook teams. Any other of these tools that they're not, something we talk a lot about because they're just there. And I feel it's the same with Copal. And of course the adoption rate is not massive, yet still with Co pilots. So there are a lot of people who haven't reached that point. But for the first movers and the people who are there and have been onboarded in the past six months, then I believe it's starting to get to that point.

00:06:03 Mark Smith
So. So what's your day-to-day role? Are you helping companies with their copilot implementation? What? What's what? What do you actually?

00:06:12 Bo Fischer Carlsen
Do so. I do a lot of different things I do training programs for companies where we specialize the trainings on how can copilot help you. I worked a lot in the banking industry. And Denmark. So how can we help financial advisors? How can we help the back office reducing Co pulp in their work process? So that's a lot of what I'm doing, but then I also take some of my history with me. So I'm also doing compliance checks, technical checks. Are you a platform actually ready for copilot? Because there's a lot of things there. And are your legal team and your compliance policies ready for cobalt? Because that's one of the massive. Bloggers is that many companies don't have basic governance at a level they want to to start using AI and even to just work day-to-day we need. So we need to improve that a lot and get the compliance experts seesaws to understand what is copilot bringing to the table. And why should we think compliance now? So I'm using that as a massive driver for doing those kind of projects. I would have loved to do a year ago, but where it was hard to find funding from the C level, sort of the purview project, the SharePoint Advanced Management and and now we have that urgency to to do that.

00:07:31 Mark Smith
The UAI act is that does that carry any implications, particularly for copilot? Not let's not go broadly into other areas of AI, but let's just stick within, you know, the when you think about the UAI act and what you would know about it. In the context of the you know, let's just call it the $30 SKU, the M365 copilot SKU. What do you think about that? And and when you're talking about things like compliance and and using that tool? How do those two things marry up for you?

00:08:08 Bo Fischer Carlsen
I think they're actually working quite well together because what the UI Act identifies as high impact scenarios and those very critical scenarios. That's not something we would do with the M365 copilot. That's something we would do with Azure Foundry or maybe agents. But what's? In the normal copilot, if you will, that doesn't really cross that line of where it has huge compliance implications regarding the AI act. There were one thing with these teams meeting recordings where you had. That uh, emotional interpretation. So what does Mark think about this? Uh podcast episode? It could do that. But now Microsoft was actually quick to allow admins to turn that on and off for the companies to decide for themselves. But as we speak, we still haven't seen EU's final. So this is what you actually need to adhere to. It should have been there in June already and may already believe so. When we see that, of course, that will tell us a lot more. But what we see from the digital data data agencies and the governments both in Denmark and in Norway is. Their strict focus is on data governance because where we see the biggest risk and where they see the biggest risk with Copal is if you have broadly shared documents in your own tenant. So it's not exposing customer data outside, it is exposing somebody's payroll information to the entire company.

00:09:41 Mark Smith
Yeah. So so part of the AI act is there's an education component that if you're going to implement AI inside your organization, you need to have trained your staff effectively on the AI tools. Does this apply to copilot in your mind?

00:09:57 Bo Fischer Carlsen
It it does. Yeah. So copilot is a generative AI tool and and it should follow the same rules and regulations. And also why? I presume I'll have a lot more to do once that really comes into play, because we need to upskill the entire country and Denmark is heavily reliant on Microsoft for most of our major companies and most of our public sector, and we need to train all those people in using AI.

00:10:24 Mark Smith
Hi. Yeah, it's funny you say that I smile because one of your politicians in the last two weeks has come out publicly saying that they're getting rid of of the office productivity suite out of that organization. And they didn't indicate that they're going to Google Workspace. So therefore, I can only assume they're going to some. No name tool for office productivity. I think it might have a the opposite effect of productivity. Is it just electioneering? Politicking, you know, type posturing or like, are they seriously going to go back into the dark ages of document creation?

00:11:05 Bo Fischer Carlsen
I really don't think so because we are so heavily bound into the Microsoft Stack that if we want to remove our entire productivity suite our entire, I mean it's identity management, it's security, it's device, it's it's everything that would be a massive project, it would take. I mean, 10s of years. It would cost billions of kroners. So I think that's. It would be very, very ambitious to undertake and I haven't seen any other state the same rather than other than this one politician. I know some people are looking into looking at some German states where they are running open source, but I also think there is a big difference of where we are coming from in those. States in Germany and in Denmark on our digitalization and what our entire course built on, I think it's a good idea of course, that Denmark should be more conscious when we build new stuff. When we buy new stuff, where do we buy that? But. Let's start there and not change the entire stack.

00:12:10 Mark Smith
Yeah, let's just head back to the training side of things and we, we, we we touched on there in regards to the AI act and you're doing a lot of training. Is the type of training you're doing. People that have never used it. They might have heard about AI and the media and they are like Ground Zero. They've they've not written a prop. They are at or are you dealing with people more of that? They're they're wanting to go to the next level of skill set development.

00:12:40 Bo Fischer Carlsen
So I'm doing both and actually enjoying doing both. I just did a series of trainings where I had eight training sessions for a company and was starting from scratch. They hadn't used chat, TPT or any other AI tools in their work day, so we started from scratch and I find that those are very, very important because copilot is not necessarily an intuitive. Tool. It's a completely different way of work. Working and just having five different copies buttons in the same screen is sort of. Where do I do what and also expectation management because we all read on LinkedIn about all the amazing things that AI is doing and we need to be able to do all of this. So expectation management and telling. So what is it actually? You can do with. I often use uh Copart in PowerPoint as the example, because if you're going into Copeland PowerPoint, which you just expectations of, what would an AI do in PowerPoint, then you're getting massively disappointed in that because that's not what Copal and PowerPoint do. But if you turn it around and actually. What is copilot then capable of doing in PowerPoint? It's capable of doing the first 80% of your presentation, not the last. 20 which is what people would expect. It would be reversed, actually, but pulling in all the information, doing a proper sort of narrative through the presentation, all those kind of things, that's what it's good at. So having that expectation management is really important in there and what I spend a lot of time when I deal with new. Uh copilot uses, but then on the other hand, also have the ones that have worked heavily with ChatGPT worked heavily with other tools or just used copilot from the start back in November 2023, I think. And there we can build into something else because the new users that will mostly be sort of app based or simple workflows. And here we can then go into. So please bring one of your more complex processes to me and then we'll work through that together. Figure out how do we tie the different Co piles together. Because that's sort of the next level. First, it's getting to understand what can the different copilots do in each. Up and then how do I then I worked with something in Microsoft 365 Copilot chat. Now I need to move it over to word. How do I do that? And then I need to make that as a presentation and so that entire process life cycle going through that that's really exciting to work with some of the more experienced users on.

00:15:17 Mark Smith
When when you're. You're training the first time users. Are you starting more in the chat experience? And so you know the M365 chat copilot experience. Sorry, let let's me just clarify there. I don't mean the non $30 SKU, I mean you're in M365 copilot. You might be accessing it from within teams, or you might be accessing it you know directly from the web browser where you have things like search, chat, the various agents that are available to you, pages, notebooks, etcetera. Are you talk? Do you do you start people there before you jump into? Hey, let's look at in the context. Let's look at it in the context of outlook or word or PowerPoint or Excel, where where, where do you where do you kind of start that training journey for folks?

00:16:10 Bo Fischer Carlsen
So that's the one of the only things in the the timeline I insist on is starting. Chat because it gives that simple interaction with an uh generative AI. It's how I learned it back when Chatt came out. Was that very simple text based interface. I didn't have to mind if it created the right slides or the right pictures, or if it formatted my. E-mail correctly or anything. Like that I just had that. Text to the LM and text back. So it's a very easy way to start working with it. And it's also where I find some of the easy Productivity Tools are sort of help me summarize this project, help me work with this, but when you then mentioned something like the agents like the analyst agent or the. Researcher agent. I shall pull them out and move them to the back of the line because it often confuses people more than it helps them work with the copilot. That and the same with with loop or copilot pages with notebooks. I still find that for people who don't use loop that then it's another layer. I need to understand it's another complexity. So also moving that a little down the line when people get that grasp of just using the LLM on my company. Jason.

00:17:33 Mark Smith
That that's interesting because you know loop I feel is almost growing up with copilot and has become a thing because the copilot, even though I use loop massively in its own right now, things that are would have done in OneNote in the past. I've stopped using OneNote entirely and I've in fact transferred a lot of my what was in OneNote. Absolute. In fact, I've just finished writing a book for Microsoft Press and I wrote that entire book. In the loop. All the chapters, all the breakdowns because it's so easy to manage, right? And from an editing perspective, it was. It was much easier and that you're not worrying about, for example, word formatting, where most people probably would write a book. In Word loop was the the product of choice. Are you then finding that you're having to sometimes level set on some of the other Microsoft products because people like wow? Is this what you can do with copilot and you're like, oh, actually, that's just a function of loop or that's just a function of forms or something like that.

00:18:41 Bo Fischer Carlsen
I'm actually anytime I go into that. If it's not just pure training, if it's sort of more an adoption project I have with the client. I start with the maturity assessment to figure out where are they on this track and that has. Several pillars to figure out both where are they with AI? Have they worked with AI already, but that's also where are they with the Microsoft 365 stack? Because I've actually met companies who don't use teams and then I mean the whole meeting experience with copilot. That's very confusing when you have to understand teams first. So we need to get that out of the way and and deal with those kind of things. And the same, I found that companies because I had a very naive approach on to start with that everybody needed these copilot or AI communities because we just see that's where you get the fastest adoption and the most efficient adoption if you can grow a a good community with the tech influencers in there. That will post about their learnings and sort of telling all customers you need to have this. But I just found that some corporations don't have a culture for that. It's impossible to build that culture. So that came into our maturity assessment. Figuring out do we build a community? Can you actually do that in this company or do we do something else? Do do we leverage town halls for getting those tech champions up and speak or do we do something? So figure out where your organization are before rolling Co piled out has a massive impact on the.

00:20:17 Mark Smith
Project you talked about maturity assessment there and a couple of weeks ago I was with a bunch of educators in the country I am in and we're talking about copilot and and and I talked about maturity assessment. But what I didn't. Expect. Is that? Maturity around something like copilot. Doesn't any differ? From how the the sea level in the organization understand their level of maturity, it differs at a department level and then it differs at an individual level. So I'm finding you can't just have. A few key people fill out the maturity assessment because you're only getting part of the picture for an organization. So when you go to run out maturity assessments and you're doing it at the individual level at the team level, at the department level, like, how do you think about if you want to capture a good snapshot of where your starting point and and how to move forward, what do you do?

00:21:19 Bo Fischer Carlsen
So it depends a lot on which part of the maturity I'm trying to assess. If it's something like, how do we train and how do we build communities around new tech, then I'll try to find the project managers that have done this kind of work before in the organization. Right now we have a lot of still a lot of those project managers that ran the teams roll out during COVID-19. I mean, they know how what works and what doesn't work in their organization for community and training. When we look at the AI maturity, we might want to send out surveys to actually have more broad picture of where are the. The different employees in their organization and then we try to get interviews with somebody from each department or from each division in the company to tell us about where are they, where are they on the processes, do they actually have defined processes they can work with on compiled? And I found that there. If I can get sort of a PA personal assistant to the department head, or if I can get one of these people that everybody will go ask for anything, they have a really good foot on the ground, they understand their department and those. The ones who can answer truthfully on their department and I will also insist on not having any management present there. I don't want anybody from the central organization there because they need to just be able to speak freely. Saying well, we are not that mature in our it. We don't know how to get these tools and and then on the training. I do always assume when I do that basic training that the users don't know anything about Copal. So I start from scratch because I found that. They're the ones who then actually do understand that they will just clap themselves on the shoulder thinking. Well, I already knew that. But if I do it the other way around, then the ones who don't understand it will just sit. With that massive frustration.

00:23:16 Mark Smith
Yes. Yeah, very interesting. I see we've consumed all our time, but just another question, when running your maturity assessments, are you just standing down and doing a Q&A session with people or are you rolling out a formal, you know a form type thing that you get them to fill out and then you know, review their results?

00:23:35 Mark Smith
How do you physically do this?

00:23:38 Bo Fischer Carlsen
So for the departmental assessment then I will send them an e-mail in advance. These are the things I'll I'm going to ask you. These are the things I expect you can answer and then I'll sit down in an interview with them to figure out what are they entering and then dig into that because most often their answers are we need to get a few.  Levels deeper in that.

00:24:00 Mark Smith
Yes, I love it. I love it. OK, before I let. You go what? What's the most recent thing you've learned? In your use of copilot. Personally like that you find like wow, that is so beneficial to me, not to your your colleagues or your customers, but to you.

00:24:22 Bo Fischer Carlsen
I I'll I'll give you 2 examples because I so so first one was and it's a silly one, but in PowerPoint I found that I needed to do spell check because I I don't spell that well and I make grammar errors. So I actually just ask Co piled in the side panel. Can you find any all spelling mistakes and grammar? Mistakes. Throughout this presentation and it just listed all the slides what I needed to change the change I needed to do and I could just go step by step through that. It helped a lot on assessing that. I felt a lack of that button that said, well, go do. But I hope that will come at some point. Yeah. And then I started playing with notebooks a lot now, and though I feel they're not there yet, they're not fully mature. And there are some levels that are just lacking. I still feel fine. That it's almost like creating a small agent, but very easily, so it's only grounded on whatever I put into that notebook and I can ask based on that. So doing that and actually starting with the researcher agent. So I worked on a new strategy, set a a couple of weeks ago. And I started with the researcher agent, turned that answer into a copilot page, and then. Put that page with all of the other supporting materials into the notebook, and then I could work iteratively towards that strategy. And that was amazing to work with. So if I can get these other tools, if I can get the researcher agent into notebook, if I can get a sort of centralized, I really want notebook to have. This is the one document I'm actually working on. This is the call. This is where we'll fill in information and just have that at glance. Then I'll be really happy because I feel there's a huge power in notebooks.

00:26:21 Mark Smith
I love it, bro. It's been so good talking to you. I feel like I've learned so much to drink from a fire hose. And in your wisdom and insight. So thank you so much for coming on the show.

00:26:30 Bo Fischer Carlsen
Thanks a lot, mark. It was a huge pleasure.

00:26:36 Mark Smith
Hey, thanks for listening. I'm your host business application MVP, Mark Smith, otherwise known as the nz365guy. If you like the show and want to be a supporter, check out buy me a coffee.com/nz365guy. Thanks again and see you next time.