

Get featured on the show by leaving us a Voice Mail: https://bit.ly/MIPVM
Simon Doy and Mark Smith explore the rapid shift from experimentation to production use of AI agents. The conversation focuses on practical agent design, Copilot Studio, RAG quality, and where automation fits better than full agents. Simon shares real client work in the NHS, small business realities, and how partners deliver Copilot adoption and agent-led transformation. A recurring theme is choosing the right level of AI, from automation with AI sprinkles to long running agent workflows, while staying pragmatic about cost, governance, and user experience.
🎙 Full Show Notes
https://www.microsoftinnovationpodcast.com/814
👉 What you’ll learn
- How agentic workflows are moving from demos into real production use
- When simple SharePoint grounding is enough and when Azure AI Search is required
- How to combine automation and agents to avoid slow, interactive AI experiences
- Practical approaches to Copilot adoption for leaders and teams
- Why token usage, orchestration, and model choice are becoming operational concerns
✅ Highlights
- “On Friday, I let Claude Opus go and build me a Teams agent overnight.”
- “The agentic side and being able to have these long running tasks is really shifting things.”
- “Customers are now getting much more open to trying things out.”
- “Copilot Studio is the first port of call.”
- “You don’t get the quality back from the agent that you need with complex content.”
- “How you chunk it is so important for RAG.”
- “Not everything needs to be a full on agent.”
- “Automation with AI sprinkles is sometimes the better option.”
- “My world is fast becoming measured by tokens.”
- “You don’t have to use the most expensive model on everything.”
🧰 Mentioned
- Copilot Studio - https://www.microsoft.com/microsoft-365-copilot/microsoft-copilot-studio
- Azure AI Search - https://azure.microsoft.com/products/ai-services/ai-search
- EmpowerM365 - https://empowerm365.com
- Microsoft MVP YouTube Series - How to Become a Microsoft MVP - https://www.youtube.com/playlist?list=PLzf0yupPbVkqdRJDPVE4PtTlm6quDhiu7
✅ Keywords
ai agents, copilot studio, microsoft 365, automation, rag, azure ai search, sharepoint, claude opus, agentic workflows, copilot adoption, power automate, tokens
Microsoft 365 Copilot Adoption is a Microsoft Press book for leaders and consultants. It shows how to identify high-value use cases, set guardrails, enable champions, and measure impact, so Copilot sticks. Practical frameworks, checklists, and metrics you can use this month. Get the book: https://bit.ly/CopilotAdoption
If you want to get in touch with me, you can message me here on Linkedin.
Thanks for listening 🚀 - Mark Smith
01:04 - The Agentic Tipping Point: Why 2026 Feels Different
01:44 - I Let an AI Work Overnight… and It Built the System
03:20 - From Demo to Deployment: Agents Are Now in Production
04:43 - Why Most RAG Systems Fail (and How to Fix Them)
06:06 - AI Has Been Here for Years—We Just Didn’t Call It AI
16:47 - Automation with AI Sprinkles vs Real Transformation
19:52 - Why the Future of Work Is Model‑Agnostic
00:00:06 Mark Smith
Welcome to the MVP show. My intention is that you listen to the stories of these MVP guests and are inspired to become an MVP and bring value to the world through your skills. If you have not checked it out already, I do a YouTube series called How to Become an MVP. The link is in the show notes. With that, Let's get on with the show. Welcome back to the MVP show. Today I'm joined by Simon from Leeds in the United Kingdom. Simon, welcome to the show.
00:00:43 Simon Doy
Oh, it's great to be here, Mark. Thanks. It's great to be back, actually.
00:00:47 Mark Smith
To be back, exactly. We were just discussing. I'm over 800 shows published, and you were around 645 last time. that you're on the podcast and we...
00:00:58 Simon Doy
That's incredible. Incredible number of shows you're doing. Incredible. Yeah, I really enjoy them. So yeah, thanks for doing that.
00:01:04 Mark Smith
Thank you. Thank you. So top of mind for you, what is it? You know, we're month three of this calendar year, 2026. I feel like there's been a shift, particularly in the AI landscape, since December. And a massive acceleration, massive acceleration, like coming from all, quarters in what I'm seeing. And yeah, what are you seeing? What, with the start of the year, what are you focused on? What's happening?
00:01:44 Simon Doy
Yeah, so, you know, I've got a small business, I think 365, and so, you know, we've really been just trying to kind of kind of, I don't know, keep up. It's moving so, so quickly. I agree with you. I think Claude is the big reason for a lot of that. Like the stuff that Claude Opus is doing is incredible. On Friday, I let Claude Opus go and build me a Teams agent. And, you know, I left it overnight, told it, gave it quite a lot. of a spec of what I wanted it to do and I just left it and in the morning it just scaffolded up sort of three projects and a solution. It's incredible. So now it didn't work first go, but a few more prompts and we really got something which we could start using. So yeah, the agentic side and being able to kind of have these long running tasks is, you know, is really, really shifting things. And customers are sort of now getting much more, sort of open to trying things out. We've been working with an NHS trust and putting an agent into, there's a big, there's a central tenant in the UK. So we're looking to put one of the first agents in the central tenant and going through that. And that just shows you that this stuff is really now coming to prime time and people are using agents, you know, in production. And yeah, that's really exciting.
00:03:20 Mark Smith
And what's the toolset you're using for agents like that?
00:03:24 Simon Doy
So for this one, it's Copilot Studio. So using Copilot Studio and you know, there are some lots of lockdown policies being applied by kind of by the administration team who run the central tenant. But yeah, it's Copilot Studio, SharePoint. to see their knowledge. I mean, this agent is a sort of a simple agent, but allowing them to get answers to their IT and SharePoint needs. But yeah, Codepilot Studio is the first port of call.
00:03:56 Mark Smith
So are you, when you say SharePoint, literally you're just giving it a site with files in it, or are you building a reg solution that with high, you know, vectorized high availability, chunking it down, that type of thing.
00:04:12 Simon Doy
So for the first one, we're just using very simple SharePoint with work IQ and it's the semantic index, but they've got another agent and that we will go down to using Azure AI search because it's just the content's more complicated, they're bigger documents, you know, and just from path experience, you don't get the quality coming back from the agent that you need. Yeah, so...
00:04:43 Mark Smith
I was listening to a podcast on RAG in regards to medical data and how you chunk it is so important between the difference of not getting enough and therefore losing context or going too refined and therefore having no leakage, linkage or connection. And, um, one of the things in my RAG system recently I have built is that it will analyze the document first and then based on the contents, it will change its chunking pattern to go into RAG based on the content. Um, because it, it's, you know, it started with, hey, let's just stand a standard chunk size. And I'm like, well, I don't like anything to be, um, you know, hard coded, so to speak. So I said, why couldn't we do this, you know, intermediate strength? And I was like, yeah. And yeah, I was doing that all with Claude too, as in the backwards and forwards and the engagement, how it could work and things like that. Yeah.
00:05:45 Simon Doy
Oh, that's really cool. Yeah. So one of the things actually with the NHS central tenant at the moment, the policies you can't use, you can't build agents for, you know, they're making any clinical decisions or anything like that. That's one of the policies. So Yeah, it'll be interesting to see how that evolves over time as we get more confident with these things. So, yeah.
00:06:06 Mark Smith
It's so funny because I betcha there's AI running across it now. It's just, it's just, you know, do you remember Nuance, the acquisition that Microsoft made about six years ago now?
00:06:18 Simon Doy
Yeah, yes, a little bit. But yeah, I don't. I've never used it.
00:06:22 Mark Smith
But yeah. So Dragon. Natural Speak was something that was, I think it was in the 80s, I first came across that product, which the whole premise of it, you could dictate and it would type, right? And of course, nowadays you're just like, like everything does that, right? But back then, it had like, I knew nothing about that tool because I had no need for it in the advancements in tech until Microsoft acquired Neurons. and then finding out they owned that IP. And it had been turned into clinical note transcription tool for doctors, because inherently doctors have a readable text, illegible, sorry, text. And so it had been trained though on like every prescription type. So when the doctor slurred their words or whatever, it knew what they were talking about because the lexicon type thing behind it was all in there. So this is what I find funny. People are like, oh, no, we can't do AI here, not realizing it's probably been going for 10 years already in your organization, but it's just become a big thing. And we're like, look at that red car over there. I'm seeing red cars everywhere because that's the media talk of the month, you know?
00:07:38 Simon Doy
Yeah, yeah. What is it? When you have that idea where you buy a car, you've never, you never see it, and then you see it everywhere, and it's just your what you zone in on. Well, I think there was the Microsoft AI tour was last week in London. It was really cool. We had Satya do a keynote there, but they were talking about Dragon at Manchester Hospital. So they were, they'd embedded it there. LinkedIn with Epic Systems, I think it was as well. So they're using it. in the UK. So yeah.
00:08:09 Mark Smith
In the US, they were in the high 90s of every medical establishment had it distributed before Microsoft acquired it.
00:08:17 Simon Doy
Oh, wow. Okay. Yeah.
00:08:18 Mark Smith
So it's a rock solid solution. And I feel that Microsoft almost have dropped the ball on that piece of IP in the acquisition, because I was doing a big deployment in Australia three years ago around... healthcare, which was Microsoft's Cloud for Health. I was implementing with a big provider and we wanted to put that in and Microsoft wouldn't allow it to be licensed into Australia.
00:08:44 Simon Doy
Oh, wow. Okay. Okay. That's crazy. Yeah. Yeah. Okay.
00:08:49 Mark Smith
I think it was the nuance of language. Nuance. I think it was the fact that perhaps they want to know that it had been fully trained on the Australian language because there's too much risk of prescribing wrong that I assume they didn't feel they had the training data in Australian to validate, even though it's English, right? But, just that, the nuance.
00:09:17 Simon Doy
That's right. I mean, I remember, mobile phones and I think Apple, and even sort of teams, I'm sure, struggled with Australian at the start and then, had more and more training data and got it better. It's the same in the UK. We've got lots of weird and wonderful accents. And yeah, I think Geordie accents up in the northeast of the UK, they struggled with those. Yeah, so yeah, it kind of makes sense. But yeah, again, it's that point, isn't it, that they They'd rather just go, no, and then do the work, which I guess, if you, depending on your risk appetite, that makes sense.
00:09:54 Mark Smith
Yeah, You've mentioned NHS. How do, you know, tell us a bit about your business. How big is it? How do you typically find customers? And because you're in the sweet spot of M365, it's a growth stage, so.
00:10:13 Simon Doy
So we're not, we're not, we're not massive. There's seven of us, yes, seven of us in the business. So yeah, we're more than a one man band, which is awesome. Like the hardest part of this is finding those leads and getting and learning about sales.And so it's been a lot of time And I've got a good team. So I've got Yvonne and Laura who do sales and marketing. And we really, we reach out to work with lots of partners. We are slightly different. A lot of Microsoft partners are kind of that traditional managed service provider. They're selling licenses, they're doing laptop, printer, M365 sort of support. But we're much more about, right, well, you've got this amazing set of tools. How do you leverage that? How do you make the most out of it? And, you know, we use tools, you know, your book that you and Meg did with the Copilot adoption, that's been really useful to help us kind of take people on the Copilot journey. But we do have these pillars, and around collaboration, communication, automation, which blends in AI and agents, and then on the data side. So we use that to kind of take customers through on a journey from sort of the beginning of their journey where we get, like, how do we make sure that they can communicate what they're doing and communicate well? Because it's really important that for a business to work, it has to have good communication, you need to have employees who are engaged in what the business is trying to do. They need to be bought into it so that they help make it succeed. And so that's what we try and use M365 to do. You know, SharePoint intranets, I know they've been around for ages, but they are good, really good tools. Tools like Engage as well for communication, recording small video snippets to kind of keep sharing what the business leaders are doing are fabulous at bringing people along on the journey so that they know how to help and how to improve the business or how they can help improve the business. Yeah.
00:12:32 Mark Smith
How do you find the love from Microsoft, being that, as you say, you're a non-traditional partner, like I've been in similar situations, Microsoft wanna fund us to do stuff with customers, but we don't, we're not ESIF green. We don't sell licenses. Do we influence them? Absolutely, we influence them from an adoption perspective, but we do not and don't want to do the transaction. Don't have anything to do with that whatsoever. And that can sit with their existing partners and things like that. But it does kind of make you an oddball when dealing with Microsoft's partner ecosystem.
00:13:09 Simon Doy
Yeah, it does. Yeah, I must admit, I don't feel like we get a lot of love, to be honest. It got a lot worse after the sort of transition from goal partner to solution designations. You know, we've tried to keep them, but that is really hard, especially in a small one. I think once you've got to like maybe 15, 20 people, keeping those solution designations are much easier because you've got the number of people. But what we do is we work with partners who can get into those ECIF funding and that is, and we leverage that.And often the partners can't do the engagements, but we can. So that works really, really well. So that's how we get around it.
00:13:53 Mark Smith
And when I was working for a UK partner, that is exactly what we're doing in that we would get the deal done on their letterhead with ECIF, et cetera, and then deliver through them. like we're bringing the deal and everything and they would take a scrape on it, and that's how it worked.
00:14:12 Simon Doy
Yeah, I guess everyone, as long as everyone wins and they, it'll get something out of it. So, but it does work well. And it also gives us access to customers, which we would have never have had access to. Do you know what I mean? So, sometimes, there's some really interesting projects that we're talking to people about, and then they've got some really interesting problems, and it's just great. I mean, that's what I enjoy is, right, you know, you're looking to achieve this thing, and like, right, you don't know how you're going to do that. But hey, I've got some ideas, we've got some ideas, this is how we'll do it, and this is our journey we're gonna go on. And I just, yeah, that's the bit I really, really love in all of this. Yeah, it's all about making a difference, isn't it?
00:14:56 Mark Smith
Are you doing much on the Copilot adoption side with companies?
00:14:59 Simon Doy
Yeah, we are, yeah. So we do work with a partner, a lady called Natasha at EmpowerM365, and she's really good at getting into the senior senior leadership team. So particularly working with the CTOs, CIOs, and she will do one-to-one training on copilot with them. And that just opens the doors. There's a book, isn't it? It's called, what's it, the $1 million proposal.And it's all about getting to the people at the top who make the decisions, who own the budgets, being able to influence them to be able to unlock opportunity. And then that's what Natasha does. And then we have a program of work and we do sort of sort of kind of basic training, more advanced training on Copilot Studio, but then go into that understanding the ways that they're working. So really delving into people's use cases, like unlocking the personal productivity, but then looking for those business team productivity where agents and kind of the bigger transformation pieces. And it's not always agents.It could just be automation. It could be automation with AI sprinkles or it could be, you know, full on agents. So yeah. I like that AI sprinkles.
00:16:24 Mark Smith
I think Donna Sakara was in Canada with her, must have been last year, and she called it, you know, the difference between sparkling wine and champagne. and automation, and it's, because so much of what I see touted as AI is automation has been around for years.
00:16:47 Simon Doy
Yeah, and I think as well, that's something some of the kind of the patterns as well is actually, because some of these reasoning models, they do take a long time to come up with a a result, like using an agent and somebody sitting there typing in and waiting for something to happen, that's not great. But maybe spinning it the other way around and automating that. So it might be something coming in, an e-mail coming in, power automate, kicking off the agent, getting a result back and then sending that result to somebody so that they don't, it might still take the same amount of time. but they don't have to wait for it, I think is a really nice way of delivering that AI sprinkles with automation in it.
00:17:31 Mark Smith
So yeah, Are you doing anything personally in the agent space?
00:17:35 Simon Doy
So yeah, I've, what do you mean sort of ones which I use every day? Yeah, so I've got, I mean, like there's a lot going on in the world, isn't there? There's lots of interesting news, unfortunately, from Middle East and things like that. But I have a couple of agents, but I have a I've nicked one of your ideas, actually, but one of them is a horizon scanning agent. So it just every morning, around about now, actually, about 6.30, it kicks off. It'll tell me the top 10 news stories, the top 10 UK, US business stories, top 10 tech stories, movies and TV. I know we've talked about this before. But they've also then got a research one, which I think you've got, where you're looking at Gartner, McKinsey, Google, Microsoft, and pulling out the research white papers just to kind of keep up to date with what's going on. And Claude as well. Claude's got some great stuff coming out of Anthropo. Don't they?
00:18:34 Mark Smith
Don't they?
00:18:34 Simon Doy
Yeah, yeah.
00:18:36 Mark Smith
I feel I've learned more about agents from Claude this year than I have in my entire career of learning about agents because They break down all the moving parts and they make them super tangible. Like there's two things I've fallen in love with so far this year. They are MD files, markdown files, just so clean, so readable, just clean, right? And I use a tool called Notepad Plus Plus. I don't know if you've come across it as a free tool, but it color codes a section so you can really read the stuff brilliantly. And then the command line interface, CLI. I've spent so much time in it because, and I'm realizing more and more, I don't need a UI. Yeah, You know, because it slows you down. I don't want to navigate a menu. I don't want to find 3 levels down as what I want to work on. CLI just takes me directly there, whether that's navigating a folder structure, whether on your hard drive or in SharePoint, it's just the speed is so much faster going through APIs. And yeah, so.
00:19:52 Simon Doy
Definitely. And that's probably better for you as well. Like, I'm getting older. switching between keyboard and mouse is not really good for your wrists and your arms. if you can all just keep typing away, then yeah, it's going to be better for you there as well. So yeah, I think, yeah, I've started, I haven't got that far in, but started looking at GitHub, GitHub CLI and starting to look into that. We do have a bit of a problem. I think we use Azure DevOps. So it doesn't. It doesn't work as well. So I think we might have to move into GitHub to...
00:20:25 Mark Smith
It's interesting because a couple of weeks ago, I did a conversation with AI around for all the type of work I do, would it be better if I went with GitHub or Azure DevOps? Because I'm licensed on both, right? So I got the choice. And it unequivocally said GitHub.
00:20:47 Simon Doy
Did it, yeah.
00:20:48 Mark Smith
Like big pro, here's all the pros of why it is and what I would recommend definitely GitHub.
00:20:52 Simon Doy
So yeah, I think, yeah, we, so about two years ago we were looking at the pros and cons and, we were going to move over. It's just, it's just such a lot of work to move over, but I think we do need to do it. And you, because you can bring in like the stuff that I like of Azure DevOps with the planning, the boards, you can bring that into Git as well. So yeah.
00:21:13 Mark Smith
It's the whole project sections, right? within it. I've not used that. But I was saying to someone today, for the first time ever, my GitHub profile is lighting up dark green over multiple days, which is, you know, you've got gray, less work, dark green, more, and the gradients. And yeah, this year I'm smashing it because I'm crafting so many things. Yeah, so it is exciting times and it's just, and I just, that's why I'm saying the quantum leap. I just feel like agents are starting to make sense and are starting to become really useful. But on the flip side of that, my little bugbear with Microsoft is it's so UI intensive, so much of Microsoft. And like one thing I didn't realize, once again, a little chat with Claude the other day, I said, and I'd never known this, that Word, Excel, PowerPoint all have APIs. And one of the real, I don't know if you know, but Claude has created Claude for PowerPoint and Claude for Excel. And everybody's raving about it. And this is the thing like I, and I had a PM on just recently in the Word Copilot team. I'm like, You guys have engineered this product over years. You know it intricately, all the features, every way you could interact with it. And yet you produce a PowerPoint that looks like it came out in the '80s. I'm like, how is that possible when you-- if we look at the average person in any of this tool set, they might use 10% in their career of the features, let's say, in PowerPoint or Excel. Now you've got a tool that could use 100%. know how to do it, know how to represent it, and doesn't. And then someone like Anthropic comes along and they bring the stuff into the super stratosphere of amazing. And so Anthropic said to me, Hey, I can give me API access. They're there for Word, Excel, PowerPoint. And I will just build your PowerPoint via API.
00:23:26 Simon Doy
And I'm like, Yeah, wow. So it's really fascinating, isn't it? Because agent 365, it's still in public preview, but the thing that I really want, right, is now that we've got an agent, it's got an identity, and I love, one of the features I love of it is that you can at mention, you know, you could create a Word proposal, right, create a Word document in it, and then you could at mention your agent as a comment and sort of say, right, you know, can you go and fill this bit out and this bit? And that doesn't, that's not possible at the moment. So we've got some Word MCP servers, but they can only create documents. They can't modify a document.
00:24:04 Mark Smith
They can- It's just, it blows my mind, right? That that's where we're at with my, as in, why wouldn't you just got engineers on to absolutely make this the finest experience possible?
00:24:16 Simon Doy
Yeah, but then you go to Claude. So I was like, right, okay. Claude has got these skills, I found all these word skills. And I was like, right, let me load that into Claude and review this proposal, create me comments where you think I should make changes and do stuff. And it did it all and it modified my proposal and it built it all out and it put all these comments in. And I was like, well, I want to link those, I want to now link Agent 365 to call into Claude and its APIs. And there's some things which aren't quite there that stop it from happening. But why is that not available in our in the M365, it's just mad.
00:24:52 Mark Smith
It blows my mind. And yeah, I just think there's a lot, I think Microsoft will get it right. You know, they tend to get it right over time, but I feel like they're just, yeah, there's a lot more to get right at the moment. Like right now, I wish Copilot, you know, how you can choose which model to use from OpenAI. Let me choose which model to use from Anthropic in there.
00:25:19 Simon Doy
Yeah, yeah, yeah.
00:25:20 Mark Smith
You know, give me that. Oh, man, we'll be off to the races, but.
00:25:25 Simon Doy
Yes, yeah. But Microsoft do seem to be shifting more to Claude, don't they? I mean, you know, you can see, you see with GitHub, Copilot, yeah, so I think, I think, you know.
00:25:36 Mark Smith
Because they don't strongly own their own though. I think they need to be fully open.
00:25:42 Simon Doy
Yep.
00:25:43 Mark Smith
You know, like I've got a bunch of agents set up and I've got eight different providers. that have LLMs into my agent pool that, and I'll be going, we're doing this task, use that one there. So I can use the best of whatever, right? Because it's all API, all token calls. And then you goes to, you know, one of the gateway providers and you can have 100, 200 different LLMs at your beck and call, like just get the dedicated one for voice or, you know, whatever. And it's just, yeah, my world is fast becoming measured by tokens. And I like it. I like it because it's something you can optimize for. And you don't have to use the most expensive model on everything. Orchestration doesn't need the most expensive model. You're orchestrating it. And so that's been a fun game with me, building out all the dashboards that monitor my hourly spend. And optimizing it has been quite a bit of fun.
00:26:45 Simon Doy
I am sure that will be, you know, a KPI on a CFO's dashboard is like, well, how many tokens have been used? How can we get that down? Yeah, yeah, yeah. Or, yeah, and then it'll be the operations directors to try and get it up. So, yeah.
00:27:03 Mark Smith
Yeah, is it, 'cause like, think more tokens, more innovation. Or maybe bad code?
00:27:10 Simon Doy
True, true.
00:27:11 Mark Smith
Right? Exactly, exactly. Hey, Simon, it's been awesome talking to you. Thank you so much for being the first appointment of your day. I really enjoyed talking to you.
00:27:21 Simon Doy
Thank you so much for having us again, and you take care and hope to see you soon in the real life. So yeah, have a great day.
00:27:34 Mark Smith
Hey, thanks for listening. I'm your host, Business Application MVP, Mark Smith, otherwise known as the nz365guy. If you like the show and want to be a supporter, check out buymeacoffee.com/nz365guy. Thanks again, and see you next time.




