For an industry that, let’s face it, isn’t known for putting itself at the vanguard of innovation, the pace at which commercial insurance companies are embracing artificial intelligence is unprecedented.

We’re now seeing AI move out of the lab and into the real world with a number of production deployments underway at large commercial carriers and brokers. While AI adoption is gaining momentum, we are still in the early days of deployed AI, so it’s natural and inevitable that we’re hitting a few bumps in the road. Some of the obstacles that are slowing progress include:

• Hesitancy about where to start or whether to join the AI race in the first place.
• Questions about how to measure AI outcomes and what success looks like for a production deployment.
• Unreasonable expectations about how—and how quickly—AI can transform the business.
• User mistrust of AI.

A common cause of these interrelated challenges is a lack of deep internal AI and machine learning expertise within commercial insurance companies. Insurers are looking to build their bench by adding AI talent, but seasoned AI professionals are in short supply globally. This knowledge gap has led to confusion around what insurers can expect from AI with respect to its application and accuracy, as well as friction between insurers and their InsurTech partners tasked with rolling out AI solutions.

If you’re reading this article, you’re probably familiar with the technical jargon used by data scientists to describe how well a machine learning model works: accuracy, precision, recall, F1 score, underfitting, overfitting, false positives and false negatives.

These technical benchmarks for evaluating AI and machine learning models serve a purpose, but they can be incredibly confusing to non-technical users. If data literacy isn’t your strong suit, the metrics above are more likely to leave you with a headache than lead you to a solid assessment of an AI solution’s potential.

AI solutions require a nuanced and rigorous definition of accuracy, as well as a thorough understanding of how they support a given use case and the overall business goals. For example, is the system more precise or more accurate, and which is the more desirable for the use case in question? The right answer may not be as intuitive as you think.

Data Models Explained

An AI model is only as good as the data it is trained on. The higher the accuracy threshold required for the business case, the more data that will be required to train the model and the more time it will take to complete the training.

Let’s take overfitting as an example. Your AI solution provider tells you they’ve trained the model and it is 98 percent accurate. Wow, that’s outstanding, right?But if the model can’t adapt to new data or changes in the data that you’re feeding it, you’d probably be better off with a model that is less accurate but more flexible. Overengineering an AI solution to achieve a few additional points of accuracycan have the unintended consequence of diminishing the model’s ability to perform well with new data when the solution is moved from testing to production.

At the end of the day, the success or failure of an AI solution, just like any other enabling technology, should be measured based on its impact on the business. You can be sure that risk-averse, growth-focused insurance executives aren’t sitting around the boardroom table discussing how they can improve their F1 score. When assessing an AI solution, ask yourself: Does the AI solution make things easier for your human knowledge workers, reduce operational costs, increase underwriting capacity and margins, help mitigate risk, or enhance the quality of the digital experience you provide to customers? These are the kinds of real business outcomes that insurers are looking for from AI—and you don’t have to be a data scientist to understand them.

Perfection or Nothing

In a recent webinar on “The Ways Machine Learning and AI Can Fail,” Brian Lange, partner and data scientist at Datascope, cautions, “Don’t let perfect be the enemy of good.”Lange’s point is that human users are often hyper-critical of machine learning and AI, and this deep mistrust of AI outcomes can prevent organizations from realizing the benefits of AI. An AI solution may be much more accurate than a human at performing a given task, but the first time the AI makes a mistake, the tendency is for users to conclude that the solution isn’t very good and can’t be trusted.

In the Harvard Business Review, Larry Clark relates an anecdote that gets to the heart of the problem: “I was working with a business intelligence executive who told a story to illustrate this problem. His internal client wanted to use a machine learning algorithm to improve his operations. His team was only about 25 percent accurate at predicting certain events using traditional analytics approaches. He wanted a machine learning algorithm that could improve their performance, with a target of 85 percent accuracy. When he was told that the machine learning algorithm could probably get him to 50 percent accuracy (twice as good as what his team could do), the client refused to implement it. Instead of seeing the massive improvement, he said, ‘Why would I roll out a solution that was wrong half the time?'”

The pursuit of perfection led this client to walk away from a two-times increase in accuracy. When evaluating AI solutions for commercial insurance, execs should keep in mind that a small improvement in underwriting margins or quoting capacity can have an enormous impact on their bottom line.

Accuracy Alone Isn’t the Endgame

There’s a party trick I like to play with potential investors and prospective insurance clients: I give them a binder, a policy and a highlighter, and then challenge them to compare the two documents and find all of the errors in five minutes. Some of them do pretty well at catching the errors and inconsistencies; others not so much. When their five minutes are up, I tell them that AI can do the same thing—with greater accuracy—in one second. You can see the lightbulbs going on as they imagine how this kind of superhuman speed could boost their underwriting capacity and free up their underwriters to focus on high-value tasks rather than the routine heavy lifting of manually checking policies.

Going back to Larry Clark’s story above, if someone told you that you had a 50 percent chance of winning the lottery, would you take those odds and purchase a ticket (or several) or walk away, leaving money on the table?

Another thing to keep in mind is that with machine learning and AI, accuracy is a moving target. The more data you feed the model, the more accurate it gets.Each time a human knowledge worker corrects a mistake, this information is recorded and used in the next training cycle. This is a continuous “rinse and repeat” cycle that ensures the system gets smarter and more accurate over time. It’s this ability to capture feedback, learn, iterate and improve over time that makes AI so powerful. If the solution’s initial results don’t seem that impressive, be patient and trust the process to get you to your goal.

Good Things Come to Those Who Don’tWait

For insurers, the pursuit of perfection may come at a heavy cost. AI implementations can take longer than you think. Most large commercial insurers will take two or three years to fully deploy and scale an AI solution across geographies and lines of business. Commercial insurance is complex. Training an AI model to understand the nuances and complexities of commercial insurance—not to mention the unique requirements of your business—won’t happen overnight.

Fast-forward five years, and commercial insurers who have not deployed AI will find themselves at a considerable strategic and competitive disadvantage. Laggards that refused to settle for anything less than perfection right out of the gate will find it very hard to catch up to companies who already have five years of solid production AI experience in the market.

If you’ve read this far and are saying to yourself, “But it’s still early days, we’ve got lots of time to get our AI act together,” it’s worth remembering the phrase coined by Kevin Kelly, founding editor of Wired, “The future happens very slowly and then all at once.” The best advice for commercial insurers that want to stay competitive and relevant in the future is to get started with AI now.