Why Most AI Projects Fail to Deliver ROI

00:00:07 Mark Smith
Welcome to AI Unfiltered, the show that cuts through the hype and brings you the authentic side of artificial intelligence. I'm your host, Mark Smith, and in each episode, I sit down one-on-one with AI innovators and industry leaders from around the world. Together, we explore real-world AI applications, share practical insights, and discuss how businesses are implementing responsible, ethical, and trustworthy AI. Let's dive into the conversation and see how AI can transform your business today. Welcome to the AI Unfiltered Show. Today, I'm joined by Justin. He comes from Charlotte, North Carolina in the US of A. All the links that we discuss in today's episode will be in the show notes. As always, Justin, welcome to the show.

00:00:58 Justin Trombold
Thank you so much, Mark. It's great to be on. Looking forward to the conversation.

00:01:02 Mark Smith
Yeah, looking forward to having a roof here for the next 30 minutes. But before we get started, tell me about food, family, and fun. What do they mean for you and your part of the world?

00:01:10 Justin Trombold
Yeah, food. Well, I. I grew up in Kansas City, lived in Texas for a long time. So BBQ always has a place in my heart, but I would say, more Tex-Mex style food. I have some ingredients to make some nachos for dinner. So I think that's my go-to go-to meals, but don't shy away from BBQ in any respect. You know, family, I have a four-year-old and a three-year-old. So family is my beginning, middle, and end of every day. And I work from home, so it's everywhere in between. And so enjoying that lifestyle and all the excitement and all the challenges that go along with that. And I think for fun, I mean, of course I have a lot of fun with my kids, but I'll be less candid than that of, I like to get together. I like to play golf, like to exercise. I like you're doing some woodworking and home improvement projects. I think if your listeners see me on LinkedIn, the first thing it says carpenter on there. And that's a bit of a joke. I've been hired for a couple projects. So I guess I'm a pro, but unfortunately, the office improvement hasn't made it for anyone that sees any video on this, hasn't made it on the list yet. So I don't have the nice background yet. But We're in a new home and so I'm doing a lot of carpentry work right now. So that's fun and stressful at the same time.

00:02:45 Mark Smith
Nice. I bet. I bet.I resonate with a lot of what you said there. My kids are three and five years old. So they occupy a lot of life and a lot of the day even. And I'm just, I've got a whole bunch of contractors here right now as We're about to kick off construction of a new home on our property. So I've got about an acre and a half. So a lot's happening.

00:03:08 Justin Trombold
Well, and I even say just one thing, when I work with my clients a lot, and then we talk about generative AI, it's important to have the why. And so as an independent consultant, one of the big whys for me related to what we're talking about is how can I build the type of business that I want, have the type of efficiencies that I want in my work so that I'm not either not present with my kids at the end of the day or after they go to bed every night, I'm plugging back in for three or four hours to get everything done. And so finding that thing, if I do this, then I can do this other thing. I think it's true for entrepreneurs, it's true for people in business, it's true for leaders. Think about what is it that you want your people to do? And I know we'll get into this, but I can't help but reflect. I was having a conversation with an individual that I know at the place where I play golf and where I exercise. And he's an insurance salesman. So he's an insurance salesman. He is exactly as your listeners might expect. But he mentioned the idea of, well, if you were living at your house and you have, you know, five different things to do, you know, why, and and there was something that you needed to do on a day-in, day-out basis, weekend, week-out basis, and you didn't have to mow your lawn, would you? And so for leaders and for people in their own business, if you don't have to mow your lawn, so to speak, why would you do it? Now, you might enjoy doing it, and so there's some nuance there, but I thought it was just a nice way to think about AI to say, is there a way to make something better, make something more efficient, so that you can reallocate even if it's just being able to think about things you wouldn't think of before. And so that's, I think, an interesting mindset going in, thinking about AI to, what are those lawn mowing activities, that you have in your business? Yeah, that you can look for those opportunities in.

00:05:11 Mark Smith
I like that reframe, because it takes it to something highly practical and it allows you to look at it. You know, often people, one of the biggest complaints I see in the industry is, what's our use case? and you've just come up with a prudent way of thinking about use cases in using AI?

00:05:30 Justin Trombold
Well, and I'd say that that's always a popular thing. And even when I do some of the client work that I do that isn't AI related, the questions we always ask people or our clients always want to ask her, what are they currently doing in AI? What are their use cases? And that, of course, has some utility to understand what's going on in the market. But the way that I like to think about it and I talk to clients about is, let's not talk about use cases. Let's talk about your processes. Let's talk about what you're trying to accomplish. And then let's go in and identify those areas where those use cases are relevant. And so why is that important? It isn't necessarily just about finding the efficiency point. And so if you give a use case and you throw in an out-of-the-box solution and you just put it in there, are you going to expect there to be improvements from that? Well, there might be. But if you didn't go and look at the process that embeds in with, there's likely going to be some other bottleneck that even if you remove it with the solution, it just might shift to a different spot. And so you have to be really thoughtful of what are the downstream or upstream implications of making something more efficient and ensuring that you have, it isn't necessarily another AI solution, But it's like, okay, now if we have a greater throughput, for instance, for front office for sales, a greater throughput and ability to screen and identify candidates for a sales team, but then they don't actually have a process to, let's say, I mean, this is an oversimplification, but you'll contact more of those people or they're not incentivized to contact more people, maybe better put, it doesn't really make much of a difference because they aren't going to change their behavior. They're going to just look at it as, well, now I can just get to my target faster and then I just work a little bit less. And perhaps that's great for them. But from a business standpoint, you're not going to see an ROI from that because they aren't going to, and you might have a couple of high achievers that just want to do more, but you just basically shifted the bottleneck to the incentive rather than the availability and quality of the leads that come into the system.

00:07:50 Mark Smith
With the customers that you're working with at the moment and your observation of the market and what's happening in the AI space, what's top of mind? What are the fears, challenges, concerns, opportunities? The landscape as we run into 2026, where do you see, the maturity level of where we, and I mean, it differs person to, organization, organization, but what's your state of play through your lens at the moment?

00:08:22 Justin Trombold
Yeah, I see what I see when I'm talking to organizations, it's, I think there's a very high level of maturity in the potential of the solution, but there's generally a very low level of maturity in the willingness to do what it takes to approach AI in a way that's going to be value creating, right? We all know, like the MIT study that came out, I think it was less than 5% generated an ROI, and some papers have come back and offered some rebuttals to that, and rightly so. But what I'm getting at is that what I see with my clients is you have organizations that are either going full steam, and they're trying to layer AI everything. And that gets back to that challenge that we just talked about of, unless you're tactical about where you put it and what the upstream and downstream implications are and what adjustments have to be made there, you're going to just put in a bunch of solutions, spend a lot of money, maybe some things are improved, but you end up with the MIT result, right? You have a lot of capital investment, very little return. But then on the other side, what I see a lot of is just fear. So it's this idea that we aren't a tech-forward company. We aren't the Microsofts of the world. We aren't those organizations. So how can we ever compete with that? And so there's a lot of just inertia in getting the process started. And so you have these two areas that in some ways converge on the same solution in terms of where they should go next. But They both are, in my view, two different types of inaction. One is inaction by over-investing and not focusing on the adaptation. And the other is inaction from a fear, this idea that a year from now, it's going to be AGI or whatever. Everybody says different things, different weeks. But I think what I typically conversations I have with them is, well, let's not look at it like that. Let's look at it as where we are today, not where you would like to be today. And if you can make something 10% better or 10% faster, and you can do that in such a way that has a measurable impact, and you feel confident about that because you looked at the process end to end, then why not just do that and start there? And then take a next step and take a next step. And for these larger organizations, it's the same thing. All right, what I'm hearing from clients and when I'm talking to, even just doing research and not working with clients, but talking to executives for some of the research I do for clients, is what do we do now, knowing that the board is starting to push back a bit and they're no longer saying, invest, invest, invest. They're saying, ROI, ROI, ROI. Well, it's the same thing. Take a step back. What is it that you can do with all these tools you have already? And what are those core business problems, those core business opportunities that can be addressed?

00:11:29 Mark Smith
The thing is, that you've mentioned ROI quite a few times, and I feel that, you know, a principle I learned years ago in design thinking was, is it viable? Does the tech back it up? Is it feasible? In other words, do we get an ROI? Is it make good business sense to do it? And then there's the human element, right? Is it going to benefit the people of the organization? Are they going to adopt it? Are they going to go in? And I've added a fourth one here, which is responsible into that type of mix. And what I have seen of the last two years and these failed, you know, lots of POCs not moving to production and things like this, is that often there was never an attempt to go, is it a business viable idea we've got here? Is it going to move the dial for us financially? And so therefore, they never moved to production because we're just throwing money for a shiny toy. I'm wondering, when you look at the, where you've seen failed rollouts, is it because of it not sitting in one of those quadrants or is it more fear from people? not wanting to get involved? Where are you seeing that sit?

00:12:42 Justin Trombold
I think it's, with that, I would have to sit and think through the quadrants for a second, but my first gut reaction is, if we think of it as like a four-square game, I don't know if that's an international game, but everybody has their own square and you're in there. So each of those are in our two-by-two matrix. Well, my view would be is that organizations or leaders or people aren't even jumping into one of those squares. So for instance, there's a conversation about what the technology can do. So it's sort of like that discussion of when you sell someone a drill, do you sell them a drill or do you sell them the hole? Right? So people are talking a lot about the drill, but what they're not doing is taking that step back and saying, like you alluded to, what is the core business objective that we want to solve? And That can be looked at in a number of different ways. It can be looked at, we have this overarching broad vision, broad opportunity, or this very thorny goal. How do we have, what do we have to have in place to tackle each part of that? Or it could be very simply, we have this one key process within a business unit, within someone's business, whatever it is. And that particular process is either a key revenue generating process It's a key bottleneck that enables you to do more work or to do other types of work. And taking that step to go and think about what that process is, go in and, like we talked about a little bit before, identify those efficiency opportunities and then making the changes that surround it. So the thorny part, the part where people are failing with POCs is they're going in and they're implementing the solution. And we've talked about before, it shifts the bottleneck to another place. But what's really happening is that leaders and people aren't wanting to answer that fundamental question that generative AI is transformation. Generative AI is a change in the way that you're working. So back to that sales example we were talking about, if you're not willing to make that hard decision, how should we adjust the incentive structure to make this work? What I'm seeing is the tool goes in, they see people that they're happy about it, it's making their life easier, but they're not seeing any results. And that could be ROI.It could be even just more leading indicators like KPIs related to the number of sales made, maybe. And then they look at it and they're like, well, and we're actually not even seeing We're not seeing more calls. We're not seeing more outreaches to potential customers. Why is that? And it's back to that point of they're asking the wrong question. They're asking about the drill rather than the hole. And so those strong generative AI outcomes that lead to ROI or lead to some benefit, we don't have to limit it just to ROI, those occur with the organizations that I work with and the use cases and case studies that I read about. for those organizations that treat AI or generative AI as an operating model shift and not some sort of tooling upgrade. It's not the drill. It's treating it as what types of holes do we want and how do we want those holes drilled for perhaps lack of a better word in the analogy. So the ability to pair the experimentation with the discipline of figuring out what it is you're trying to do and what outcomes you're trying to measure and really understanding when you look at the outcomes, how should you interpret that data? Like what's the way that we should look at and how should we, how can we tell if it's a failure of the tool or a failure of some other decision that we either didn't make or didn't want to make that would enable the value creation there? So I think that's the sticky part that I'm seeing right now. And so back to your quadrant, The square is on the ground.The 4 squares are on the ground for each of those. And I think people are jumping into that feasibility bucket, but just from the technology angle, not from the organizational angle and what has to be true.

00:17:05 Mark Smith
Yeah. You know, and what I took from what you're saying, you're saying people, rather than trying to use the AI tools to create incremental gains, they need to see that it is a, if done right, it's actually a new way of working. It's a new way of doing business to really open it up. And that's, so one of my challenges that I've been going over in my mind in the last week or so is, ROI a red herring to a degree? In that I don't ask, do I have ROI on electricity? It's A fundamental part of my infrastructure to run my company. And I'm thinking that, have we considered AI a new piece of software tech? rather than a fundamental shift in the way we're going to do business probably forever. And people are still just treating like it's another piece of software.

00:17:58 Justin Trombold
Yeah, I think that's, I like the way that you framed that. And what comes to mind when I'm thinking about, ROI for this idea popped in my head that, I had a conversation with the small family shop, they work in, private wealth, asset management, real estate, and so forth. And there's this interesting discussion of, well, let's say we don't have the discipline to measure the metrics or measure those returns. Like we just don't, we don't have the infrastructure or the culture in place that does that. It's like, okay, well, that's That doesn't mean you can't look for these solutions. And perhaps, Mark, what you said as well is maybe it's not such a simple process to say that if we get more leads and we change the incentive structure, now they can sell more. And we can problem solve that one, but it's, well, what happens if I just make my organization overall more efficient or I make myself overall more efficient? Well, what has to be true for that to happen? Let's take the example I use with my kids out of it, although that's a good illustrative example that you kind of see the outcome, right? You can feel when that happens, but you have to have some vision of what it is that you're able to do that you couldn't do before. And so as an example, you could not demonstrate, or you have an inability to demonstrate time savings or whatever the improvement is that you're looking for in the solution. But let's just say part of the longer term vision of your organization was, we need to have our people spending more time looking for deals, like with this small company, and less time just doing whatever they do on a day-to-day basis. It's like, well, if you set up the generative AI structure, and you did it in such a way that you were pretty confident they were going to be able to work more efficiently. And then at the same time, you do a careful institution or application of these objectives or these processes to go and find more deals and think more strategically. You can then observe, even qualitatively, like is my team now thinking differently, right? Are they spending their whole day with their heads out of the computer? Or are they out now having lunches, having coffees, and talking to people? And so you don't have to be so rigid with it. And if it is a larger scale outcome that maybe like 6 different changes enable, You can't take that very simple process view, but you have to think carefully and say, here's this thing I want everybody to be doing that they don't have time to do now. If I make these six changes, are they now able to do this thing they were putting in place that they're saying they didn't have time to do before? And then it becomes kind of a binary answer. You know, sort of that concept of you know it when you see it, if something's going well. And you may not be able to trace it back to generative AI. But in some cases, particularly for smaller organizations where you're not beholden to a lot of different metrics, that's often a good enough answer, right? Yes or no, like I'm seeing my people do more of this thing that I want them to do, but you have to very clearly define what that thing is, right? And you have to identify those places that generative AI can be used to free them up from those lawn mowing type activities. I don't know if that answers your question, but that's what came to mind when you were describing it.

00:21:40 Mark Smith
While you were talking, I've got a few concepts that I want to share with you and then get your take on them. So I feel that there's two parts in an organization. Either they want to increase productivity from output of their personnel, or they want to innovate. The challenge with innovation, most organizations have an inbred part of their culture not to innovate. They have KPIs, they have OKRs, they are told, follow this process, don't deviate, do your job, hit these numbers, everything's fine. We're not looking for innovation. AI potentially opens up a massive innovative frontier to us like never before that we're not necessarily getting now inside the organization. We haven't set up a culture inside that organization for it. A colleague of mine was talking to a representative from a large law firm in the US. Their specialization in law was in business law. And one of the things they did, they had a piece of a matter that they would do for organizations that would take about 40 hours for a lawyer and their staff to carry out. And they would charge around 25,000 US for this piece of work. They applied AI.

00:22:57 Justin Trombold
Sounds like a nice setup. Yeah.

00:22:59 Mark Smith
Right? And honestly, they had years, like that work, the type of stuff they were doing was just rinse and repeat, rinse and repeat, you know, as in so very lucrative business. AI comes along and challenges them on how could we do this now? lawyers work on a model of charge by the minute, right? So when you say, let's make you more efficient, they're like, well, hang on a second, you know? You're going to reduce our time billing. So what they did is they took that though, and this is where I love the innovative thinking. They were able to apply AI to the process and reduce it by 50%, the effort. So down to 20 hours that they could do it. In fact, they even, the efficiencies were larger than that over time. But what they were able to, as part of it, as going through multiple iterations of this, The same lawyer's time could do about four of these in the same amount of time, these matter type cases. So what they did was half the price to the market. The customer won, now we're getting it now 50% reduced on what it was costing a year ago, but the law firm were being able to, I think, four times the number in the same amount of time. And that's what I talk about innovation. When you really relook at your business and go, hang on a second, are the levers of yesterday not the levers of today or tomorrow?

00:24:22 Justin Trombold
Yeah. this is what's, and it reminds me of a couple of conversations. This was a conversation with a smaller firm and a larger firm, but a similar theme because it's these levers of productivity and innovation are often looked at separately. And I think the fundamental mistake people make is they think about innovation in AI as the AI solution innovating something. And that's a very complex thing to wrap your head around, because there are too many unknowns. Like, what type of innovation? Like, how do you have a tool that then... But what you're talking about is, as you can be more productive within a given business, and you're confident that the service you're providing is of value, and it would be differentiating to be able to do that, you know, faster, cheaper, better, whatever it is, you can then have, it's innovation of the business model, but it isn't radical innovation, right? But it's an ability to then differentiate in the market, do more of what you can, you get more customers, you grow your customer base. An analogy or similar aspect of that is, with this concept that I've been kicking around with some of my clients is that, you know, generative AI gives people permission to innovate. And so what does that mean? It's like, take that same lawyer and maybe they're now not the same person now isn't just grinding and doing four times the work, but perhaps they're doing 40% more of the work or whatever it is. And now they've recognized that it's been holding them back in the past, the law firm, that their junior associates I'm not a lawyer. I'm just throwing this out there.Haven't been able to do anything aside from those billable hours and cranking out that work product.They can now spend their time either doing other things or thinking about ways in which the firm can innovate. So you get this freeing of time that you can, instead of just thinking about a task, you can actually think about what it is you're trying to accomplish. You can think about relationships. And so an interesting, I think his name's Daniel Coyle, I'm blanking on it, but it was a conversation on a podcast called Econ Talk, which isn't for everybody, but it's a great podcast. But they were talking about what it does to your mind when you're task-oriented. And I think we can all, without getting into the details on what they talked about, I think we can all associate with, well, how do you feel, how well do you think about and think about things, have creative solutions, think about relationships when you're in that task mode. And it's crank, crank, crank. I know for me and for most people, and they talked about that in the podcast, it's a very constraining mindset when you're just thinking produce, produce, produce. And so being able to free up time where you're not in that mindset can have miraculous unintended benefits. And so these organizations that I'm talking to, and this was specifically from some interviews I was doing for a separate research project, of the CIO just reflecting philosophically on just the behavioral change that he's seen in his people now that they're, they aren't just doing all the time. Like they're actually having time to reflect and think and build relationships. And that comes with all kinds of serendipitous benefits that you can't You can't necessarily plan for that on the front end, but you can set up a culture where people can be more innovative as well, as well as the more type of productivity innovation that you were talking about.

00:28:14 Mark Smith
I like it. Justin, if folks want to get in touch with you, what's the best way for them to do that and kind of, you know, how can you help them?

00:28:25 Justin Trombold
Yeah, you know, I'll just say, how can I help? You know, just, I'd love to just provide information like this on these podcasts, but in terms of how I can help people. So I work with large and small businesses to help very specifically find some of those value-creating generative AI opportunities, whether it's at the process level, the broader business level, and put together a path of what needs to be true. So what are the simplest operating model changes that have to be in place? How do you upskill your people? How do you get them to be users and smart users? So that's what I do. And if people want to learn a bit more about that, I have a little toy version, let's say, of a generative AI readiness diagnostic on my website. It's about six or seven questions. You answer it, gives you just a general idea of where you or where your organization stands. And then, you know, if people want to fill that out, I'm happy to continue a conversation with any of your listeners that would be open to doing that.

00:29:28 Mark Smith
Awesome. I like it. We'll make sure those links are in the show notes. I'll also get off you after the show, the link to that podcast episode, because we'll put that in the show notes as well, because I'm intrigued to have a listen, because I think it's a really good lens to have on, you know?

00:29:42 Justin Trombold 
Yeah, I'll say you either love EconTalk or you don't. It is in my view. So yeah.

00:29:48 Mark Smith
But that whole thing around task orientation, why it resonated with me is I can be very task orientated and I'm like, hang on, what can I learn from this? So let's get that in the show notes as well. But Justin, thank you so much for coming on and sharing your insights.

00:30:04 Justin Trombold
No, Mark, it was a pleasure. And hopefully your listeners enjoyed it. And I look forward to hearing from any of your listeners and keeping an eye on what you're up to here in the coming months and years.

00:30:16 Mark Smith
You've been listening to AI Unfiltered with me, Mark Smith. If you enjoyed this episode and want to share a little kindness, please leave a review. To learn more or connect with today's guest, check out the show notes. Thank you for tuning in. I'll see you next time, where we'll continue to uncover AI's true potential one conversation at a time.