From POCs to Production: Ship AI That Sticks

00:00:07 Mark Smith
Welcome to AI Unfiltered, the show that cuts through the hype and brings you the authentic side of artificial intelligence. I'm your host, Mark Smith, and in each episode, I sit down one-on-one with AI innovators and industry leaders from around the world. Together, we explore real-world AI applications, share practical insights, and discuss how businesses are implementing responsible, ethical, and trustworthy AI. Let's dive into the conversation and see how AI can transform your business today. Hey everyone, welcome back to AI Unfiltered. Today's guest is from the USA. Full links are in the show notes as always. Jim, welcome to the show.

00:00:51 Jim Spignardo
Nice to be here.

00:00:53 Mark Smith
Good to have you on, Jim. I always like starting with food, family, and fun. What do they mean to you and your wonderful part of the world?

00:00:59 Jim Spignardo
Jeez, well, food is a big part of that. I think when I look at The fun in my life, it's typically surrounding food, whether it's traveling to a new place, it's always finding local places to eat and enjoying cuisine I may not have tried before. You know, we pride ourselves in my family, even my kids who are now kind of grown and out of the house of trying lots of different things. And we actually have a menu on our our pantry door that kind of most people come over to our house and see that and wonder, how do you make this stuff every day, every week? So family, I live in upstate New York and I have three children and I've been married for close to 30 years now. And yeah, The extended family has kind of spread out and are in various parts of the US, but we've called this area our home for more than 25 years. And as far as fun is concerned, I have a lot of interests. You probably find that with people in tech all the time, just very curious. I love gardening. I've been doing it for quite a while. I love growing my own food from seed and being able to reap the benefits of my labor, which sometimes can be quite extensive. But also I'm into home automation. So if you were to look at my router at any time, there's probably close to 100 devices connected to my network, automating pretty much everything in my house, including the light over my head.

00:02:39 Mark Smith
I identify with so much of what you just said. I'm big into my veggie garden, growing a centropic forest, and everything in my house is automated. Every switch, every PowerPoint, everything. So I'm right there with you on that. Tell me a bit about your background. You're director of AI enablement. What's it all about?

00:03:04 Jim Spignardo
Yeah, so that really is a role that came to me in the last eight months out of my work with my organization, ProArch, being a strategic consultant, working with customers. When ChatGPT dropped over three years ago now, I really took a strong interest, going back to my curiosity, right? And took the charge within our organization to really try to transform our business into an AI-first company. And that included, as being a Microsoft partner, looking for ways to also assist our customers as well in that journey. So we began experimenting and really just figuring out how we could ingrain artificial intelligence into our processes. And then about six or eight months ago, our CEO, who is a very forward-thinking, innovative individual himself, said, we really need someone who is at the helm and is the point person in our organization. to really make sure we can drive that adoption. And I'll use a Microsoft term, become a frontier firm. And that's when he kind of proposed the position to me. And at first I was a little anxious and like, I'm not even sure what that means, but it's been an incredible experience so far. And I just absolutely love working with not only our internal teams to help them, but also our customers as well.

00:04:42 Mark Smith
 I like it. Microsoft published a couple of papers on frontier firms, and they're definitely leaning into it in this FY across the globe. When you think of a frontier firm, if someone, a layperson asks you what that was, how would you describe it?

00:05:02 Jim Spignardo
Mostly as marketing fluff right now, but if we think about it from the perspective of really analyzing everything we do within an organization and how we modernize those processes. How do we take the things that are mundane and dull and distracting away from our users and unlock their true potential as employees of our organization so that we can continue to be competitive and grow in our industry. That includes things such as looking at data, which really is the jewels of almost any organization now. And so if we are going to automate and use artificial intelligence to improve as a business, you have to look at the data 1st and make sure that those processes that surround the data, the data itself is in a good state. to be able to take advantage of modernization. And at the end of the day, no organization can truly call themselves a frontier firm. It's really more of a goal than an ending, right?

00:06:23 Mark Smith
Yeah. It's, I don't know, just as we're talking there, and I'm wondering if it sounds very similar to digital transformation with a new name.

00:06:33 Jim Spignardo
In a way, in a way, with an infusion, the side infusion of AI, right?

00:06:39 Mark Smith
Yeah, And I like the concept of not just trying to improve a process with AI, but if you were to do this process from scratch with an AI lens, would be a lot different than just kind of bolting a bit of AI on the side of something, right? And the impact of business?

00:06:58 Jim Spignardo
Absolutely.

00:06:58 Mark Smith
Two things I've noticed in the last 12 months in business, is 1, a lot of POCs being built that don't move to production and analysis by paralysis, right? There's this old thing that has happened in organizations, perhaps, you know, you've heard of FOMO, there's a version of it that people worry about ****** up and whatever they do, is it going to be crippling to their business, you know, going down this? And so they don't move forward and take advantage of the opportunity. Maybe that's right now. How are you addressing that? What are you seeing?

00:07:39 Jim Spignardo 
Yeah, what we're seeing is that, to your point, a lot of organizations don't really know how to get started. And our approach to that is, think about the business problems that you have. What are you trying to solve for? And if you don't have an understanding of that, if you can't inventory those challenges and struggles, then all the AI and all the technology in the world is really not going to get you off the dime. So it's important that we approach it from that perspective and lens first. Clearly, you have to understand the technology and the underpinnings of the technology as well. But once you can identify those problems, As A consulting firm, we can come in and say, well, this is where you could use AI in this situation to solve for this problem. But you really have to map it out really well. You have to kind of describe, you know, who is affected by this problem? What is the impact that this particular pain point causes the organization? And what do you deem as success, right? What do you want to see at the end of the day, the outcome to be if you were to leverage this technology? And then you kind of have a story. You've written a story where you can say, okay, we're going to plug in and we know exactly what our goal is, where our North Star is, and how, you know, we'll plot out and figure out how to get there. And don't be afraid to iterate, but iterate fast, right? If it's not working, look at a different solution and choose a different pathway. But what typically can help organizations is start with something small. You know, don't try to boil the ocean. Figure out, you know, a specific role or persona within the organization, something that causes a lot of friction, and demonstrate some value really quickly. to kind of unlock people's thought processes around, where else can you leverage this technology? And what we've noticed is that there's kind of a lag period between being introduced to the technology until people finally start to get it and the light bulb goes off. And then all of a sudden it seems like the floodgates are open. And all of the ideas come rushing out and you almost have to hold them back to say, okay, slow down. Pick a few good things here. Let's do them well and let's get them done properly. And let's make sure you also have a good, solid foundation for continuing this journey into the future.

00:10:20 Mark Smith
I see a lot of companies starting with Copilot, kind of M365 Copilot is that starting point, is that You can't create catastrophic damage inside a business, and yet if adoption happens, the impact can be phenomenal. How do you, is that one of the tools that you're using with businesses or, you know, getting them on their AI journey?

00:10:44 Jim Spignardo
Yeah, absolutely. That's really my main focus is around coping on adoption. And I pretty much developed the solutions and the solution plays and bundles for our organization that we plug into our customers. And when I started working within ProArc as well, it was the initial thing that we focused on, focused on making sure that we can increase the literacy of our teams, give them some specific use cases to use these tools in, and also identifying the metrics we want to track. That's very important as well. And we were very intentional and methodical. took the crawl, walk, run approach to make sure that people could first make sure that they're comfortable with the technology. And then over time, for example, I had my marketing director call me today because she wanted to work on copilot agents. She had developed a couple agents. She wasn't getting the specific outcomes she was looking for. She was getting pretty good outcome. But we just went through some instruction sets to show her how she could tweak this, tweak that, give the agent the different resources to work on. And it was amazing to think that. If this was the same time last year, I was not getting those questions from our team members, right? But now they are, we are, because they're starting to understand what is possible. And really, it's kind of a snowball effect, and that momentum begins to build and build and build.But only if you put the right processes in place and you have good governance and good structure. If you put these tools out there and say, okay, we bought you one, now go work harder and do better, right? That's not a strategy. That's not a strategy at all.

00:12:36 Mark Smith
So what are those ingredients when you see businesses that do lean in and adopt, kind of when you're taking a new business on that journey through your, you know, smart start program, what are those initial things What are those core ingredients that you need to get in place with a company to know that you're setting them up for success with AI?

00:13:03 Jim Spignardo
Yeah, the first thing is it's not an IT initiative. That's one thing we have to be very clear of. If all we're doing is talking to the folks in IT, we pretty much know that's going to most likely fail. We've got to bring everybody to the table. We have to have executive sponsorship and buy-in. And then we kind of start with the education process. We just expose them to what is possible with these tools. And part of that is also helping them establish good governance. So we run them through a data risk analysis with their M365 data state. So we can show them, yeah, these tools are available, but you want to make sure that All these things that you've always said you should be doing, you're actually doing now. You're checking your permissioning. You're making sure that you're not sharing things externally. You're setting up policies and data labeling to protect certain data. And at the same time, giving them some tips and tricks around development of use cases. A lot of times when we're first coming into an organization that's very new to this, they don't even have the language to describe how to develop use cases. So it's very important for us to show them examples and give them ideas of, you know, this is an example of how you could leverage this technology within this role, within this industry. And then they start to get it and they start to pick up on it. We also love to leave behind a knowledge hub for them. something that they can come back to. So we build a center of excellence in SharePoint and we fill it up with all types of educational materials, but also guidance for the IT group because they're going to have to manage the security around it and be aware of every new feature that comes down the pipe and know whether they want to enable it or disable it.

00:15:01 Jim Spignardo
We also work with them to develop a good AI usage policy so that they can clearly state to their team members what is acceptable, what is not acceptable with this technology. And that gives the users more confidence in using it when you're just setting, establishing the expectations. And then lastly, we train them on how to measure the impact. and keep things moving, developing a champion program within the organization for folks who are going to be responsible for the success of the program. When we leave an engagement, if that's not something that we haven't identified within that organization, who's going to carry the torch when we're gone, most likely we tell that organization, here's a couple options. You can hire us and use our chief AI officer services, or you need to find someone in your organization who's going to take the mantle. We're also very big on making sure that there's some sort of counsel or committee within that organization that is cross-functional. It can't just be the IT group and the developers. So in our case, I had our AI governance council, but it consists of our CISO, our CTO, people from finance, executive leadership, from our digital engineering team, and a bunch of other groups. So there's a cross-functional representation on that team, and we can decide as a group, you know, what are the initiatives we want to focus on, and how do we spend our time as wisely as possible? Because let's face it, we could spend 60 hours per individual on these things, but we all have other jobs too.  So those are kind of the core components that, in my opinion, lead to success for most organizations.

00:17:04 Mark Smith
What are you seeing on the, you know, because you started by saying that it's not about, you know, it's not an IT project, and I 100% agree with you. In fact, where I've seen that happen, adoption tends to be very low. What about the executive layer or the executive branch of the organization from the CEO down? How is important, or what's the impact that you're seeing of their involvement and their buy-in and their messaging across the organization and how that impacts the success when it comes to adoption?

00:17:43 Jim Spignardo
Absolutely critical. But it's no different than any other type of initiative, right? If you don't have consistent messaging from the top leadership, folks are not going to take this as seriously as you would like. So, you know, in our case, our CEO put out an open letter to the organization, mapping out the vision for where we want to go as an organization. Our governance council It tends to be as transparent as possible. After we meet every month, we put those notes out to the organization, letting them know what we're working on and giving them an opportunity to provide feedback. So, but it's like anything else. If you try and do a security initiative and say that everybody has to use pass keys or go to passwordless authentication, and then you find out that the directors and the managers aren't using it, but everybody else is forced to, it becomes one of those things like, there lacks the authenticity and the accountability. So.

00:18:48 Mark Smith
Just one last focus point on the executive suite. If you were to rank just maybe just a gut feel of what you're seeing, which executives from, are you seeing most bought in to adopting AI inside an organization? Which would help you rank them.

00:19:08 Jim Spignardo
Sure. I do typically see that the CTOs and the CIOs are the ones that, obviously that's their bailiwick. That's the thing that they live in. If you have a progressive CEO and they kind of can define what that is, I mean, we're a special case. We're a technology company. We were doing AI before ChatGPT. but we were able to quickly identify where our opportunities were. But I would also say to a certain extent, the CISO is uniquely motivated to be interested in this technology, not necessarily from the standpoint of using it, but understanding what impact it has to the business from a security and risk perspective. So there's typically a lot of interest there in making sure that, hey, even if we're not using it, even if the CEO hasn't told us what direction we're taking, we need to get in front of this because shadow AI is real and we can't just pretend it doesn't exist. With folks who are the CFOs and the CEOs, they're the ones that are hearing about it and fearing that they're not doing something because they feel like that's the FOMO, right? That everybody else is doing it, so we should be too. But I would argue that sometimes they're not the most knowledgeable on what that really means to the business. But they just know, well, we're supposed to be doing something.

00:20:45 Mark Smith
Yeah, You mentioned an AI council there. What type of rhythms would you see a successful AI council operating under? what is their frequency? What things should they be addressing? How should that run successfully inside an organization?

00:21:04 Jim Spignardo
I think initially, you probably want to meet on a monthly or more basis, maybe even bi-weekly. One of the things that we did was to initially establish our charter and our vision as an organization. And I think that's really what leads us in everything else we do. So that's something we made absolutely clear to the rest of the organization to kind of allay a lot of the fears some of them may be having. Is this stuff going to replace me? Is that the actual goal of our organization is to reduce headcount by using automation and AI? And so we made it very clear, that's not our intention. Our intention is to use this and actually give you these tools as an expense to the business that we're going to track and try and recognize value from, but to be a force multiplier, not essentially to say, well, now we don't need you anymore because whatever you did before can be done by AI. Apart from that, One of the other things that we identified really early on, besides an AI usage policy, which we continually evaluate. So we update ours at least annually, sometimes more frequently. But we also wanted to inventory our skill sets and the initiatives that were somehow already out in the environment. and making sure we add visibility into every department where they're using AI, whether it's embedded in their line of business applications already, or they're talking to vendors about using it in some other capacity. That cannot exist in a silo within that business unit.

00:22:51 Jim Spignardo
It needs to be brought to the council. It needs to be inventoried. It needs to be analyzed. We need to make sure we do a proper security review on it, a proper business value review to see if what we're using or what we're doing actually makes sense. Because again, there's only so many dollars in the budget, and there's only so much time on the clock. So those are kind of the main objectives. And then long term, we also are trying to look at our processes across the board and figure out, are we positioned correctly as an organization to take ultimate maximum value from these tools? Meaning, do we have good integration between our line of business applications? Do we have a way to tap into that data with AI? And are our processes sound? Do they work today without AI? And if they don't, we need to make sure that we can first shore them up and ensure that we have good data integration and also a good understanding of how the work that we do in one business unit affects another business unit upstream and downstream. Because if we can do that, we're really going to be able to truly maximize the benefit of AI.

00:24:15 Mark Smith
You mentioned a champions program. How do you identify the people inside an organization that are going to be your founding members of a champion program? What is the process when you take a new company through this and make the recommendations and What's your identification process of those, it may be the first five people, maybe it's only one person, but how do you identify those in an organization?

00:24:44 Jim Spignardo
When you're looking for those individuals, sometimes it's just a matter of asking, right? Whoever wants to raise their hand is typically better than someone who is volun-told to do that position. But we all know that there's certain individuals within our organization who are the early adopters for any technology. And most likely, they're already using this as well. I would also say, too, that sometimes you have to wait a little while to see who surfaces, right? Who kind of pops up from the masses and demonstrates that they have aptitude in this area. But in the absence of that, organizations also can't be afraid to hire out, right? If we can't find the talent, and within, then we should be looking for that talent to bring on board. I've been very clear with our CEO, going forward, we have to make sure that every job description involves some language in there of AI literacy and both aptitude and desire to continue to use this technology in their everyday workflows. But if that can't be the case, so let's assume an organization, it's a small organization, they don't necessarily have someone that they've identified as a champion. That's where at ProArch, we really pride ourselves in having our chief AI officer services that we can come in and play that role, at least temporarily, until such time as an organization can nurture someone internally or identify someone externally. And we purposely make that engagement three, six months, or one year, And our goal is to work ourselves out of a job. If after one year you're still wanting to hire us, then you are not taking this seriously enough. Our goal is to get someone in there so that we can get out of there, essentially.

00:26:46 Mark Smith
I like it. Where are you at with the buzz around AI agents?

00:26:52 Jim Spignardo
You know, I'm, like I said today, we're really drinking pretty, heavily from the AI hose right now to the point where we spent some time training our users on how to actually build agents. We did a little bit of a workshop and I was amazed at how many individuals have taken that initiative to go out and build something themselves to the point where I was mentioning our marketing director set up some time with me today because she built a few agents to help our inside sales team. And I was astonished at how far along she had gotten in building that without a lot of extra effort. Now, obviously, it takes some expertise and a little bit more finesse to get it really where you want it to be. But overall, as an organization too, we're very committed to beginning to build agents, full code agents and low code agents to really start to lock or be able to tap into our line of business applications. we're using Fabric and data warehousing to bring data in and then attach Copilot on the back end to start to really gauge some great insights and powerful abilities out of that. So yeah, I would say I haven't taken a look, I just got access to our M365 admin center to look at our agent usage because that's something I need to report on a regular basis to our leadership team. But I think I'm probably going to be pretty shocked at how many agents are out there now that I didn't even know existed.

00:28:30 Mark Smith 
So true. My final question is based on something you've said twice in this conversation, which is digital literacy or AI literacy, but particularly the word literacy. How do people develop their literacy in this area of digital slash AI area? Like how How do you bring the literacy level across an organization up as a whole?

00:28:59 Jim Spignardo
Yeah, I don't think we can expect them to do it on their own. That's just absolutely the case. And we decided very early on that we needed to create a program where we onboard teams to our way of doing things. And we're very specific in what we want to see as those outcomes. But we also build a community within our organization. So I run a user group for our Copilot users. Every month, we pick a different topic. We do a very deep dive into that topic to explain how to build an agent, for instance. This week, we're going to be doing using the new pre-built workflow agent from Microsoft to show them how to use that. And we ensure that we keep them up to date on all the features and capabilities. So we have a Teams channel where we put out tips of the week. We have an AI news briefing that I actually collect news resources across the web that specifically pertain to our business, our organization, what the impact is. And We're just in the infancy of actually putting together a very formalized AI onboarding program, which will not only be for new employees, but it'll be for any employee that gets assigned A copilot license, that they will be required to go through all this training and understanding of the products, with even potentially even a quiz at the end to prove their competency. And that's not going to stop, again, because And there will be folks who will just go out and learn this themselves. They've always done that. They're the people who read the manual for every single hardware or compliance they buy in their home. But most people... are very anxious about this and they aren't necessarily comfortable with it. So you have to expose them through this, through repetition and through experimentation. I think experimentation is critically important in a safe environment where they can fail and be comfortable with it.

00:31:06 Mark Smith
I love it. Jim, it's been a pleasure talking to you. I've learned so much. Thank you so much for coming on the show.

00:31:11 Jim Spignardo
Thank you for having me. Appreciate it.

00:31:13 Mark Smith
You've been listening to AI Unfiltered with me, Mark Smith. If you enjoyed this episode and want to share a little kindness, please leave a review. To learn more or connect with today's guest, check out the show notes. Thank you for tuning in. I'll see you next time, where we'll continue to uncover AI's true potential one conversation at a time.