Financial services — insuranceMultimodal vision-language models & RAG

Multimodal AI that checks vehicle photos and videos against the policy before a person sees them

An insurer’s team checked every customer photo, walk-around video and VIN plate by eye, and bad ones were only caught at review. We built a multimodal verification module that rates each submission, reads the VIN and checks it against the policy — plus an agentic-RAG upgrade to its Messenger chatbot.

A customer submits vehicle photos, a 360-degree walk-around video and a VIN close-up. A multimodal analyser rates each for clarity, lighting and angle with a written reason, confirms it shows what was requested, reads and decodes the VIN, checks the details agree across images, and compares the submission with the policy record retrieved from a knowledge base of the insurer’s policies and rules. Submissions that pass reach an operator with the analysis attached; those that fail go straight back to the customer with feedback on what to reshoot. Rejected submissions feed back as quality data. Results along the bottom: 90%+ quality assessment accuracy, 95% discrepancy detection, 70% fewer improper submissions and a 90% first-time approval rate.
The customer fixes a bad photo while they are still uploading. The team only reviews evidence that passed.

Project snapshot

Client
An insurer selling and servicing vehicle policies through digital channels
Industry
Financial services — insurance
Business function
Policy verification, underwriting operations and customer service
Challenge
Every vehicle submission — photos, a 360-degree walk-around video, a VIN close-up — was checked by hand for quality, completeness and consistency with the policy record. Blurry, badly lit or incomplete submissions were caught late and sent back, and the customer chatbot could not answer beyond scripted topics.
Solution
A multimodal verification module: a vision-language model rates each image and video for clarity, lighting, angles and coverage with a written explanation, confirms it shows what was requested, reads and decodes the VIN, and checks the details agree across images. Retrieval over the policy knowledge base flags discrepancies with the policy record. Customers get real-time feedback as they upload. Alongside it, an agentic GraphRAG layer behind the existing Messenger chatbot, synced with the helpdesk.
Result
90%+ accuracy on image and video quality assessment, 95% detection of discrepancies between submissions and policy records, 70% fewer improper submissions and a 90% first-time approval rate — each workstream delivered in four and a half weeks.

Key outcomes

90%+
Accuracy on photo and video quality assessment
95%
Discrepancy detection against policy records
70%
Fewer improper submissions
90%
First-time approval rate

The client

An insurer whose evidence arrives by phone camera

An insurer selling and servicing vehicle policies through digital channels. Before a policy is confirmed, the customer photographs the vehicle, films a walk-around and captures the VIN plate. Most of them also talk to the insurer through a Messenger chatbot.

Both channels worked. Neither scaled with the team behind them.

The challenge

A team checking photos one by one

Every submission was reviewed by hand — for quality, for completeness and against the policy. A blurry VIN, a missing angle or a dark video was only spotted when someone opened it, and then the customer had to start again.

What each submission needs

3
evidence types — photos, video, VIN
360°
walk-around coverage
1
policy record to match

Meanwhile the chatbot handled scripted topics well and everything else poorly, so those questions also landed on people.

Constraints

What we had to design around

  • EvidenceA photo is only useful if it is sharp, well lit, taken from the right angle and of the right thing. The system had to judge all four, and say why, not just return a pass or fail.
  • VideoStill images were not enough. A walk-around video had to be checked for full 360-degree coverage, and a VIN close-up had to be legible enough to read.
  • The policyA good photo of the wrong vehicle is still a failed check. Every submission had to be compared with the customer’s policy record, using the insurer’s own policies and rules as the source of truth.
  • The customerFeedback that arrives after the customer has gone means another round trip. Quality had to be judged while they were still uploading.
  • One front doorCustomers already talked to the insurer on Messenger, with a helpdesk behind it. The chatbot upgrade had to live there, not in a new app.

Our approach

Let the model look, let the policy decide

Vision-language models are now good at describing what is in a photo. They are not a source of truth about a customer’s policy. So we split the job in two.

A multimodal model does the looking — quality, content, the VIN, consistency across shots — and writes down what it saw. Retrieval over the insurer’s own policies and rules does the comparing. And the verdict reaches the customer while they are still uploading, because the cheapest reshoot is the one that happens straight away.

The solution

From a phone camera to a verified submission

  1. Ingestion — A pipeline for vehicle photos and videos with format and quality validation on arrival, 95%+ data capture and secure storage at 99.9% availability.
  2. Multimodal analyser — A vision-language foundation model rates each image and video for clarity, lighting and angle, with a written explanation. It confirms the shot shows what was requested, reads the VIN, decodes make, model and build date, and checks those details agree across every image.
  3. Policy retrieval — Retrieval-augmented generation over a knowledge base of the insurer’s policies and rules. Each submission is compared with the policy record and discrepancies are flagged for the operator.
  4. Customer interface — A chat-style upload flow with guided prompts. Customers see the quality verdict as they upload, and receive their policy details and documented damage report once verification is complete.
  5. Service chatbot — Behind the existing Messenger bot, a triage agent routes each question to a query-resolution agent working over a GraphRAG knowledge graph or a real-time data agent calling live APIs. A responding agent replies; guardrails, caching and feedback logs run in the background. Conversations sync with the helpdesk.
The video and photo verification module. Inside a core module, a multimodal analyser takes three inputs — vehicle video, quality assessment and a policy mapper — into an insight generator built on a multimodal language model, which draws on three understanding components: context understanding, video understanding and policy understanding. Its output passes to a data processing stage that sorts submissions into valid videos, valid photos and valid VIN, each with a pass or fail mark. Valid items flow to an operator. A feedback loop labelled gather quality data runs from data processing back into the core module.
The analyser combines context, video and policy understanding. Only items marked valid reach the operator; the rest feed the quality loop.
Two workstreams of four steps each. Photo and video verification: half a week on ingestion with 95%+ data capture; two weeks on multimodal quality assessment at 90%+ accuracy across clarity, lighting, angles, 360-degree coverage and VIN close-ups; one week on policy retrieval with 95% discrepancy detection; one week on the customer upload interface, reaching a 90% first-time approval rate. Service chatbot: half a week on a data pipeline into a knowledge graph; two weeks on a GraphRAG agent with 90%+ accuracy and 95% of answers in under five seconds; one week on real-time data agents refreshing sources in under five minutes with under 1% errors; one week on Messenger and helpdesk integration with 99% message success.
Both workstreams follow the same order: data, model, grounding, then the channel the customer uses.
4.5 weeksper workstream, from data pipeline to a customer-facing channel

Responsible by design

Four rules for AI near a policy decision

  • Explain

    Every rating carries a reason.

    “Sharp, well lit, no blur” or “damage visible, no lighting issues”. An operator can see why a photo passed, and a customer can see why it did not.

  • Ground

    Policy facts come from the policy record.

    The model does not recall what a customer is covered for. It retrieves it from the insurer’s knowledge base, at 95% retrieval precision.

  • Escalate

    The model checks; a person approves.

    Valid photos, videos and VINs reach an operator with the analysis attached. Discrepancies with the policy are flagged, not decided.

  • Learn

    Rejections become training signal.

    Failed submissions feed back into the analyser as quality data, so the checks sharpen on the cases customers actually get wrong.

The third rule shapes the rest. The analyser does not approve policies. It makes sure that what an operator approves is complete, legible and consistent with the record.

Results

What changed for the verification team

  • 90%+ accuracy on photo and video quality assessment — clarity, lighting, required angles, 360-degree coverage and VIN close-ups, each with a written explanation.
  • 95% detection of discrepancies with policy records, with policy details retrieved at 95% precision.
  • 70% fewer improper submissions, because guided prompts and real-time feedback catch the bad shot while the customer is still holding the phone.
  • 90% first-time approval rate, so most customers verify in a single pass.
  • A chatbot that answers beyond its script — over 90% response accuracy, 95% of queries answered in under five seconds, 99% Messenger message success and helpdesk sync within five minutes.

The team stopped sorting photos and started approving submissions that were already right.

Beyond the numbers

What else changes

Every rejection now comes with a reason the customer can act on. “The VIN is out of focus” gets a better second photo than “please resubmit”, and it arrives before they have put the phone away.

And because each rating is explained and each discrepancy is logged, the insurer has a record of what was checked and why — useful long after the policy is issued.

If your team still checks customer photos by eye

The first question is which checks are about the image and which are about the policy. That split is visible in a week of rejected submissions.