Designing and building a safety harness for our insurance LLMs

Today’s language models are more than capable of talking to customers about insurance. But capability is only the beginning. To be ready for customer use in a regulated financial environment, you need a safety harness around the model - a continuous system of monitoring, explainability, and protection that ensures every interaction stays compliant, auditable, and trustworthy.

1. Continuous monitoring and benchmarking

Insurance language is regulated language. That means every response must be demonstrably fair, clear, and not misleading. We have built an evaluation framework that continuously tests the models we deploy against thousands of real insurance questions to detect failure modes such as missing disclaimers or language that risks crossing the advice boundary.

This framework powers the Compliance Risk Index, our benchmark for model safety. It measures performance across four dimensions: evidence and retrieval, policy logic, communication standards, and the detection of vulnerability or complaints. Each dimension is scored at scale, giving us a measurable view of how the models we deploy perform over time, and how they compare with off-the-shelf alternatives.

‍

2. Explainability and traceability

We believe all model outputs should be explainable. That means that we must be able to attribute model responses to ground truth source material. 

It is not enough to generate an answer and hope it is right. In our harness, every response is built on evidence: the model first retrieves the exact passages of policy wording that answer the question, and the response is generated from those passages.

Because that evidence is captured alongside the response, every statement can be traced back to the exact text within policy wordings it came from, and independently checked against it. This means we can provide industry leading levels of explainability and traceability, whichever model sits at the centre.

3. Retrieval accuracy at scale

To answer customer queries with correct and relevant information, the model must be able to draw from a dynamic knowledge base with thousands of policy documents and supporting materials.

Traditional retrieval-augmentated generation (RAG) systems rely on vector embeddings. However, these embeddings introduce brittleness:

• Chunking strategies must be retuned as documents change
• Embedding models can have blind spots for domain-specific terminology
• Retrieval quality degrades when new policy types are added

Rather than bolting a brittle retrieval pipeline onto the side, we give the model direct search tools. It issues precise text-pattern searches over raw policy documents, composes multiple searches to handle complex queries, and importantly self-corrects when initial searches don't yield useful results.

This approach delivers higher retrieval accuracy than embedding based approaches, even as the number of documents increases. 

This means that when customers ask questions, they can be confident that the responses are accurate, even when comparing many policies at once.

4. Hallucination detection and output guardrails

Generative models can sometimes fill gaps with plausible but unsubstantiated information. In insurance, unsubstantiated, potentially false information can lead to customer harm. We need to be able to detect this, and ultimately prevent any hallucinated content from being exposed to the consumer.

A secondary check, independent of the model that generated the response, has the sole purpose of detecting hallucinations. It does this by confirming that any information contained in the output message is in fact contained in source material that the model referenced at the time of generating the response.

If hallucinated content is detected, the output is flagged and the message can be regenerated or the conversation can be escalated. These controls ensure that every response is grounded in verifiable, in-context evidence, ensuring the consumers can trust the model’s outputs.

The result: safe, confident deployment

Together, these mechanisms form the safety harness around every model we deploy. They make the language models behind products like Insurance Companion not just powerful, but safe to deploy directly to customers, delivering compliant, trustworthy experiences at scale.