Navigating Pitfalls on the Pathway to Innovation

Neil Hodge


December 1, 2021

A man in business attire faces a wall, on which is a maze drawn in chalk leading to four lightbulbs, one of which is lit up.

Innovation is both a buzzword and a lifeblood for businesses, arguably one of the most critical components of ensuring business longevity. However, actually executing on innovation can be fraught with danger, especially in an economically turbulent market. History suggests that businesses that invest in innovation during a crisis outperform their competitors during recovery. However, many companies have experienced notable failures as a result of misguided attempts to change what they do, how they do it, or what they deliver, whether it is a classic example like New Coke or something more recent like Google Glass.

 “Great firms don’t rest on their laurels—they are always looking to see how the market is shifting, looking at what customers want, and finding the best way to source the right product or service for customers ahead of their competition,” said Richard Swales, chief risk officer at online payments company Paysafe. “Innovation is about truly understanding your risk appetite, your competitive pressures and understanding how you will be protecting your existing lines of business while innovating on new or existing products or services.””

In many respects, innovation initiatives are like any other projects: They require strict discipline in terms of planning, implementation, management, monitoring and budgeting to ensure they succeed. However, according to the Project Management Institute (PMI), the overwhelming majority fail to deliver what they set out to do, and around one in six are deemed total failures. This is partially because boards and senior management often neglect to clearly define the project’s scope, its goals and how to measure progress. They also fail to build in contingencies so that the plan can be modified or scaled down if necessary.

Monitoring the Big Picture

Even after the initial plan is approved, companies must constantly ask themselves whether the innovation strategy will still be the right one by the time it is finished. Goalposts often move in line with customer expectations and market forces, which means the project’s goals may also need to shift. As a result, the organization must have the capability to change the project’s direction and scope, if necessary, and be receptive to internal challenges to stop it from going off the rails.

“For many companies, these projects are deeply personal. It can be all too easy to become so engrossed in the product that you lose sight of the bigger picture,” said Peter Bracey, managing director of accounting firm Bracey’s Accountants. “But they need to ask themselves: What is the competitive landscape? Is there a viable market for the innovation that will give you return on investment? It is imperative to have a team around you that will not only support and drive the project forward, but that will also challenge.”

Companies need to remind themselves of the ultimate goal behind any innovation and ask themselves exactly what is needed and why the innovation is taking place, said Alyona Lamash, director and head of the risk management practice at consulting firm Exactpro. Trying to copy a competitor’s innovation strategy can also present its own set of challenges and risks, she warned. “Does the innovation solve a technological problem, or it is rather driven by market trends? In case of the latter, the risks are that you may implement processes that mimic the success of the others, but which, in reality, only create a disarray of workarounds and interoperability issues for yourself,” she said.

Another common problem is that companies assume that the project will pay off and that positive change will always produce positive results. However, what constitutes “success” is often poorly defined—if at all—while common metrics and benchmarking can be largely ignored. “Companies need to be able to define what they want the end results to be and know what success looks like,” said Chris Gill, director of risk and insurance markets at facilities management software vendor Over-C. “Too many organizations go into innovation projects without really knowing what the process entails, what the scope is or should be, and what the actual goals are supposed to be.”

He added, “Once you are clear about what you want to achieve, you will have a better chance of determining whether these objectives have been met.”

Conversely, some businesses have unrealistic views of how innovation works and the benefits it should deliver. “If innovation is seen as finding ‘the one big thing’ or viewed as a silver bullet to solve longstanding problems, then the chances of finding that inside a large established organization are extremely small,” said David Collins, managing director at IT consultancy First Derivative.  

According to William Chia, head of the school of business and law at the London School of Business and Finance in Singapore, many companies tend to appraise the project from an internal perspective using internal criteria. They often do not take an “outside view” that requires project leaders to benchmark their companies’ projects against objective and historical data derived from similar initiatives elsewhere. This skewed focus, he said, “perpetuates the planning fallacy where risks are underestimated while benefits are exaggerated.”

To achieve a more accurate forecast of any project, he recommends that companies adopt “reference class forecasting,” which allows project leaders to more accurately forecast the risks and probability factors before deciding if they should embark on the innovation project in the first place. Admittedly, though, finding a similar class of projects to benchmark against can be difficult depending on how innovative and unique the project is.

Cost Considerations

Better forecasting also helps control costs. One reason projects fail is the inability to get the financials right. When resources run out, projects stop. According to PMI research, over a quarter of project leaders say inaccurate cost estimates are a major factor in scrapping business strategies.

“The single common denominator is that no one ever gets the quantum correct of how much investment is needed for an innovation project,” Bracey said. “Planning ahead and ensuring you have enough visibility of your finances to know when you might run out of money is key for business continuity.”

Part of the problem is that many companies automatically—but incorrectly—assume that innovation involves a radical overhaul, new technology and deep pockets. As a result, they assign large budgets from the outset and continue to make further investments during the course of the project to address what are essentially planning problems.

“Many companies risk falling into the misconception that their innovation must be disruptive (requiring new business models) or radical (requiring new technical competence), and that it will incur costs, investment and effort on a mammoth scale,” Chia said. “However, research has shown that most profits are created through routine innovation by leveraging existing business models and technical competence.”

To keep costs from escalating, many businesses can benefit significantly from not aiming for the “perfect” solution and instead looking for a “collection of little wins rather than a big bang,” Gill said. This could include finding ways to reduce paperwork so that goods or services can be brought to market more quickly, simplifying internal processes to reduce costs and improve turnaround times, or enabling customers to pay through more channels to make purchasing easier.

Regulatory and Compliance Risks

While some projects encounter cost overruns, others can face regulatory obstacles. Some organizations—particularly those that want to be seen as industry disruptors—get so hung up on whether they can develop an innovative product or service and get it to market first, they forget to question whether it is actually compliant. Financial services firms have been guilty of such tactics in the past, for example, when selling subprime mortgages. Now, app developers and other technology firms that use personal data as a commodity are coming under closer scrutiny from data protection authorities all over the world.

Contrary to some arguments, regulations do not stifle innovation. Instead, many experts assert that putting compliance at the heart of any innovation project can provide focus, help reduce project failure rates, and ultimately save money and reputation.

Having the risk management and compliance functions closely involved from the start can help minimize risk in the innovation process, according to Nigel Jones, co-founder of data protection technology provider The Privacy Compliance Hub. This increases the development team’s awareness of risk and instills a culture of continuous compliance. Without their involvement, he said, “a company will either release noncompliant products or have their product launches delayed at the last minute by compliance professionals, who will then be (unfairly) blamed as barriers to innovation.”

Mark Gallagher, a former Formula One executive who is now a managing director at consultancy Performance Insights, has seen first-hand that strict regulations and compliance requirements, as well as reduced budgets, can promote successful innovation rather than stifle it.

Since the 1990s, the rules and regulations around safety in auto racing have become much stricter, as has their enforcement. These changes have increased the effort required to ensure and monitor compliance. However, some in the sport saw this as an opportunity to engage with regulators to find out what technical changes they could make to the cars to gain a competitive advantage.

Gallagher said this approach saved time, money and heartache in the long run. “The sport is not awash with cash,” he said. “No one is throwing money at ‘trial and error experimentation’ anymore where the only way to test if innovations worked was to put a driver in a car.” Instead, racing teams began looking at the rules and thinking of ways they could make improvements while remaining in strict compliance. They also consulted with the regulator before spending money to make sure they had interpreted the rules correctly before moving onto the next stage.

The Importance of Buy-in

Perhaps the surest way for any innovation strategy to fail is if it lacks management leadership or buy-in. If managers at all levels are not fully behind an innovation project, experts say it will most likely be doomed.

Management reluctance can be an “innovation killer,” said Greg Murphy, vice president at Instanda, which provides IT products to the insurance industry. Industries that have so far been less impacted by disruptors may also question whether they need to innovate as rapidly as others. “Innovation can mean rapid, drastic change and the fear of the unknown can put boards off from getting behind schemes that are deemed very high-risk,” Murphy said. “If they are not really committed to the project by being actively involved, they are unlikely to champion it, especially if it runs into snags.” 

Collins agreed that internal resistance is a key barrier to successful innovation. He believes many organizations have a “frozen middle”—the top of the company may want to change, but middle management is “too vested in the existing structures,” viewing change as a weakening of their power base. Consequently, they stifle attempts at innovation or disruption.

As such, it is important for the organization to have a culture where innovation can flourish. Once again, management needs to set the tone. “If risk appetite is defined and the company encourages and supports innovation, then employees will be empowered to innovate and support the process,” Swales said. If this is done well, he added, “news will naturally travel, success will breed further innovation, and creative minds and successful people will be drawn to the company because they will know that it will be a safe and supportive environment to innovate.”

It is also vital that management does not outsource responsibility for the strategy. Because many companies assume that all innovation is either technology-based or technology-led, they often assign leadership, project management and implementation to the IT department. These arrangements frequently lack management oversight or regular assistance from other relevant departments, such as risk management or internal audit. And even when these functions are all involved in the project, they are frequently operating in silos rather than as a single unit.

“IT is not the ‘innovation hub’ of a business,” Gill said. “IT enables operations but it does not lead strategy—that is management’s job. However, too often, senior management takes a back seat whenever technology is involved and key innovation strategies often fail as a result. It is crucial that senior management—with boardroom input and backing—leads and takes ownership.”

Ultimately, companies also need to recognize from the outset that their attempts to innovate are potentially high-risk and that there may come a point when the plug must be pulled. Unfortunately, this is not always as easy as it sounds.

Once the innovation process is underway or when top management has already committed to the change, it can be very hard to kill an innovation project, even when it is doomed to fail. This is because they have “become so enthused and committed to the strategy that their belief becomes infectious and unstoppable,” Chia said. Management may also be hesitant to cut the cord as the company has already invested a lot of capital and labor into the project.

“This blind faith, or optimism bias, might ultimately cost the company a few extra millions of dollars with nothing to show for it at the end,” Chia said. “Companies, therefore, need to institute stringent monitoring and governance mechanisms that will allow the project managers to sound out early warnings to discontinue the project if it is evident that the project is never going to succeed.”

Neil Hodge is a U.K.-based freelance journalist and photographer.