How we built an AI solution for GEICO CDP

99% Accuracy, no fine-tuning, no vendor

By Sachin Jhunjhunwala
November 3, 2026

Overview

We set out to build an internal AI chatbot for GEICO's Customer Data Platform (CDP). We evaluated internal/external options, ran a full vendor proof of concept, and eventually chose to build in-house.

The result is CDP AI-Copilot: a production chatbot that lets GEICO teams ask plain-English questions about complex customer relationships across policies, claims, and quotes, and get answers in seconds. It uses GEICO-approved LLMs with governance, access controls, and PII masking built in.

This post covers the problem, our build-versus-buy decision, parallel POCs, the in-house architecture, what shipped for CDP's transactional graph layer, and lessons learned.

The Problem

CDP data was powerful but only limited to programmatic access

GEICO's CDP stores rich, interconnected customer data in a graph database: policies, claims, quotes, and the relationships between them. The data was powerful, but access was limited to API calls.

The challenge wasn't the data. It was fast, easy & secured access to that data for internal teams to resolve customer issues.

Internal customers wanted to ask questions like:

• Fraud: “Find all claims for user ID 1234,” “Visualize every graph connection for this customer and lets drill in,” “Show the shortest path between two people.”
• Support Agents: “Give me all policies, claims, and quotes for this email.”
• Marketing: “Show users with only an auto policy and boat quotes but no boat policy for cross selling campaigns.”

Challenges Before AI-Copilot

• Limited adoption from technical barriers: Internal users needed API integrations or deep schema knowledge to write complex GraphQL queries, limiting CDP access to a small group. Goal: grow from tens of users to tens of thousands.
• Slow time to insight: Simple answers could take days or months, leading to missed opportunities with marketing, fraud insights, bad customer support experience, etc. Goal: move from hours or days to seconds.
• Hard dependency on engineering: Customer teams relied on engineering to answer routine data questions, increasing request volume and slowing delivery. Goal: remove this bottleneck & load.

The AI Solution: Why was this hard?

1. Graph data is different, and answers had to be fast. Enterprise tools target relational/warehouse data, not graph traversals, relationship search, shortest paths, or multi-hop queries at terabyte scale. No native graph support, fast GraphQL generation or interactive visualization meant no CDP solution until we pivoted to AI.

2. AI requires more comprehensive governance & data catalog. We lacked enhanced role-based masking, a centralized metadata catalog (attributes, ETLs, graph vertex/edge, lineage, freshness), and clear per-field/per-user return rules for AI. Without this catalog and these guardrails, no AI solution could exist in CDP.

3. There was no dedicated product, design, web or AI expertise in team. The same team handled product discovery, web design, architecture and full-stack development with no prior AI skillset. Building AI with limited expertise while playing all other roles was a massive challenge and led us down the obvious path of looking for an AI vendor.

4. Training data had to be created. We couldn't build an AI solution and test it on live production data. CDP's graph data needed a purpose-built training GEICO environment with months of customer data, plus privacy/security reviews and approved model access before any AI work could start.

5. Accuracy was non-negotiable. A chatbot that returns wrong customer data is not just a bad experience; it is a business risk. We needed a rigorous way to test accuracy across dozens of query categories clarifying intent and reducing token costs both at the same time.

Evaluating Every Option

We didn't jump to build. We evaluated every viable path—from existing GEICO solutions, a specialized external vendor, and an in-house approach—against the same core requirements.

Option 1: Existing GEICO AI solutionsDecision → no go

We spoke with internal teams already building AI chatbots and found that most relied on external vendors. None could support CDP's graph-centric requirements, terabytes of data, near-real-time expectations, and complex relationship queries.

Option 2: A specialized external AI vendorDecision → 5 month POC

We identified an external vendor with the most promising graph fit. They supported bespoke GEICO development, on-premises deployment, full data sovereignty, and a no-cost five-month proof of concept. The tradeoff was dependency risk, a limited enterprise track record, longer timelines, and an estimated ~$200K annual cost if contracted.

Option 3: Build in-houseDecision → ran in parallel post 3 months vendor POC

Building in-house gave us full control over the product, models, governance layer, and roadmap. The risks were real—limited AI expertise, more engineering bandwidth, and no vendor safety net. But the upside was stronger—native graph support, integration with any data store, GEICO-approved models only, and no vendor dependency. After three months of the vendor POC not reaching the accuracy we needed, we ran the in-house path in parallel.

The Dual-POC Strategy

Rather than betting on one path, we ran vendor and in-house POCs side by side using the same query sets, success criteria, training data, and customer feedback loops.

Vendor POC

For the vendor POC, we stood up a GEICO training cluster with six months of historical data, prepared graph schema and validated custom integrations, trained the vendor agent, and tested enterprise readiness across deployment, security, and performance. We ran weekly syncs, pushed fixes, and retested quickly.

In-house POC

In parallel, the team wrote scripts to use a GEICO-supported AI model directly, ran the same query sets against the same training data, built a UI, added a semantic intent layer, improved prompting, and created orchestration logic.

Query Sets Tested

• Direct lookups by key or indirect attribute plus single-hop traversals off each entity by partition key, email, phone, claimId, QuoteId, etc. (~100 queries). E.g.: “Can you find the person with ID kfdsjfkd..,” “Look up policy 434343...”
• Complex multi-value and multi-node traversal. Multi-hop traversals (2+ node types) and multi-value batch inputs in a single request (~25 queries). E.g.: “What other policies does the policyholder of policy 232323 have” (policy → person → policies, 2-hop), “Show me everything—all policies, quotes, and claims for person 98893434” (fan-out across three entity types).
• Aggregation and count queries (~30 queries). E.g.: “How many policies does person 343k4jjk have,” “How many vehicles are insured on quote 4343jkj..”
• Relationship search and existence checks with open-ended “show me everything related to X” searches, and yes/no boolean checks (~20 queries). E.g.: Relationship search: “Show me everything related to policy 114222342,” Existence checks: “Is person 43322323 also a driver on policy 111111..”
• Edge property queries that depend on properties of the relationship (edge) rather than the nodes—role and active/deleted state (~10 queries). E.g.: “Who is the driver with role OWN on policy 232222,” “Show me only the active drivers on policy 999909090, not deleted ones.”
• Visual natural language graph queries (~15 queries). E.g.: “Show me the shortest path between person 43433 and person 243433,” “Help me visualize all claims of a person with id 434343,” “Expand/Remove this node.”

The Decision: Build In-House

Criteria	Vendor POC → 5 months	In-House POC → 2 months
Query accuracy	~77% (eventually got to 90%)	>99%
Time to market (V1)	~6 to 9 months	~3 months
Annual cost (over 10 years)	$1M–2M	Amortized over few eng
Graph compatibility	Partial	Native
Product ownership	Vendor-dependent	Full
Governance / PII masking	Limited	Built-in
Intent orchestration	No	Yes
Ease of adding features	Vendor roadmap	Team-controlled
Interactive Graph visualization	No	Team built
Multi-DB integration	Limited	Graph + OLAP + metadata

The vendor solution improved during the POC but never closed the accuracy gap. The in-house path gave us higher accuracy, faster iteration, lower cost, full ownership, built-in governance, native graph visualization, and flexibility for future agentic and OLAP access.

Solving the Hard Problems

• Governance from scratch: CDP holds sensitive PII, so the team built role-based access controls and data masking directly into the copilot as a core architectural requirement which is reusable for future CDP AI products. We also had to build with the necessary guardrails in place so that AI didn't run amok (rate limiting, restricting number of graph query hops to one level deep, validating results, etc.).
• Metadata as a source of truth: There was no consolidated metadata store (attributes, ETLs, domains, graph vertex/edge, stores, lineage, freshness, etc.) for graph data. The team built a new CDP metadata catalog as source of truth for data access with APIs and governance controls, so AI queries could be ‘grounded’ in accurate, current CDP metadata.

  Grounding in AI is the practice of connecting an artificial intelligence model's responses to verified, external material instead of relying on its internal training data. We had to make sure AI answers were grounded within boundaries of what our metadata supported.

• Training infrastructure in weeks: The team decided to snapshot 6 months of real production data into a new training cluster, masking all the PII and leveraging that to test the AI solution. Working with GEICO Site Reliability Engineering (SRE) and the AI Organization, the team provisioned approved models, implemented ‘token’ limits, and stood up the historical data training cluster while completing privacy and security reviews—all within a few weeks.

  An AI token is the smallest chunk of text that an artificial intelligence model reads, processes and generates. Most commercial AI services charge money based on how many tokens you send (input) and receive (output).

• 99.99% AI Accuracy without any LLM training for less $$

  

  • 1. ‘Ground truth’ query set: We created a bespoke query set (~200 queries) mapping natural language questions to real GraphQL queries, drawing on current APIs and infrastructure and manually generating queries for all customer use cases not directly supported by existing APIs. A lot of NL questions for each GraphQL query were created by asking the LLM to think of potential variations.

    A ground truth query set is a curated collection of test prompts paired with verified, accurate answers that serve as a benchmark to evaluate search engines or AI systems. [1, 20]

  • 2. ‘Prompt engineering’ as a discipline: We created a set of predefined rules for all queries, feeding them plus graph metadata schema (YAML: attributes, vertices, edges, properties, partition keys) to the LLM to generate GraphQL. Every failure (wrong traversal, wrong edge direction, missed partition keys, bad syntax) became a new rule or schema fix to get comprehensive rule coverage for all query paths through offline and iterative ‘prompt engineering’. The same approach was leveraged to improve output formatting of results without any LLM training or ‘fine tuning’, saving $$$.

    Prompt engineering is the art and science of designing, refining, and optimizing text inputs (LLM prompts) to guide AI models to generate the most accurate and useful responses possible.
    
    Fine tuning is the process of taking a pre-trained LLM and training it further on a smaller, task-specific dataset to specialize its behavior, style, or knowledge—which costs money.

  • 3. Validation loops drove accuracy: We manually reviewed generated queries, explain paths and results, refining schema and prompt until outputs were consistently correct across the test set. The 200-query set functioned as a ‘tuning loop’ such that examples didn't ship with the runtime prompt, but the lessons they taught did.

    A tuning loop in AI is a continuous, automated cycle where an application or model observes real performance data, evaluates results, diagnoses errors, and updates parameters (like prompts, retrieval settings, or model weights) to improve accuracy over time.

  • 4. ‘Few shot RAG’ & embeddings for intent gathering: We passed the curated query set in advance through an embedding model (nomic 1.5) and stored similar queries in a vector DB (PG vector). At runtime the user's question retrieved similar examples from the vector DB (using cosine similarity search with top-k similar queries) that were fed to the LLM alongside the schema + rules in the prompt, which is classic ‘few-shot prompting’ done as ‘RAG’. Only with sufficient similar queries did we run the actual resulting query, thus getting the right intent & answers along with sufficient guardrails. If no similar queries were retrieved, it led the LLM to ask further clarifying questions.

    Few shot prompting is an AI technique where you give a large language model a small number of examples (usually two to five) within your prompt to show it the exact pattern, tone, or format you want.
    
    RAG: Standard large language models (LLMs) rely only on static data they were trained on, which can lead to outdated answers/made-up facts, known as hallucinations. RAG fixes this by connecting the AI to trusted outside sources, such as company documents, databases, or live web searches—or even including extra sources of data in the prompt, which is what is referenced here.

  • 5. Intent as the routing layer for our ‘MCP tooling’: Confidence-scoring whether a question mapped to a supported query let us branch into the right MCP tooling—from graph exploration, metadata search, general Q&A, document lookup to guardrails for off-topic input—such that the orchestrator didn't treat every input the same way.

    Model Context Protocol (MCP) tooling is a standardized open standard that allows artificial intelligence models to securely discover, trigger, and execute external functions—such as querying a database, reading local files, or calling APIs—through a universal interface.

  • 6. ‘Orchestration’ matured beyond the POC: LangChain powered an early Python POC for our AI solution, but production needed tighter integration, so we moved to a custom Java Spring Boot layer that classifies intent, invokes the right tool, generates and executes graph queries, summarizing results conversationally.

    AI orchestration is the practice of coordinating multiple AI components—from models, agents, tools, APIs, and data—into a single workflow that runs reliably in production.

• Graph visualization without a vendor: Interactive graph visualization was requested for years, and external tooling would cost ~$200K annually. The team built it directly into AI-Copilot—including relationship exploration, node drill-down, shortest-path views and future Fraud case management integration—by converting the JSON graph result of a query into a React representation with LLM help and rendering that on a UI interactively.
• Customer-driven iteration: The solution and test query set were built via feedback from Fraud, Marketing, Data Science, and Engineering teams. Fraud teams got a month of beta access pre-launch for early feedback. Slack feedback channels and bug bashes drove pre-release polish.

The Final Architecture

CDP AI-Copilot couldn't follow a simple “user asks, LLM answers” pattern. Customer data spans a graph database, metadata catalog, and (future) analytical stores like Iceberg and Snowflake.

Queries range from point lookups to multi-hop traversals to (future) OLAP aggregations, with every response passing through governance + PII masking before reaching the user.

We chose 7-step orchestration over raw prompting—classify intent, ground in metadata, select execution path, then generate/execute the query.

The data layer stayed modular, supporting graph/metadata lookups today, with future room to expand into OLAP and agent-to-agent workflows.

Entry points: Human users access AI-Copilot through the CDP Self-Serve UI, while GEICO AI agents can access CDP through MCP for agent-to-agent workflows.

The seven-step orchestration loop:

Step	Action	What happens
1	Access control	Governance layer verifies the user's role and data entitlements before any processing begins
2	Reason / Intent	Intent service classifies the query using a vector DB of similar past queries and refined LLM prompting with grounding
3	Plan action	Orchestrator decides the execution path: operational (real-time graph lookup) or analytical (OLAP aggregation)
4	Execute action	Query engine routes the request to the appropriate specialized agent and data store
5	Observe result	Raw results return from the graph, metadata catalog, or analytical store
6	Mask result	Governance applies PII masking rules based on the user's role before any data is exposed
7	Answer	Formatted response—text summary, table, graph visualization, or CSV export—delivered to the UI or calling agent

What We Shipped

The first release delivered a production-ready AI copilot for CDP's transactional graph layer.

• Users can ask natural-language questions across people, policies, claims, quotes, and relationships; receive answers as summaries, tables, CSV exports, or graph visualizations.
• Users could also use the chatbot to explore customer connections, node details, and shortest paths interactively in the UI alongside the chat.
• Users could also get answers about any metadata that lives in the system—from attributes, their sources, domains/LOBs and any other metadata.
• Right architecture to support MCP and analytical OLAP queries in the future.

Lessons Learned

1. Build-versus-buy should be tested, not assumed. The vendor path was the responsible place to start, but the POC gave us the evidence to make a better decision. The rigor of both paths against the same data, query sets, and success criteria moved the conversation from preference to proof. Building in-house was not just a cost decision. It gave the team control over costs, accuracy, governance, extensibility, user experience, and future integration with OLAP data, metadata, and GEICO's agent ecosystem.

2. Governance must be designed into AI systems from the beginning. For customer data, accuracy is only one part of trust. The system also has to know who is asking, what they are allowed to see, and how sensitive data should be masked before anything is returned—along with the right guardrails stopping AI from running amok.

3. You can build AI products without any training or fine-tuning, saving $$$. AI accuracy didn't come from fine-tuning or training an LLM. It came from offline iterations with refining rules, manually injecting schema/rules in prompt until they nudged the model in the right direction, validated offline. Every failed query taught us something about the product, the data model or a user intent we hadn't captured yet. Guardrails came from confidence scoring at the intent layer, with only high-confidence queries getting routed to MCP tooling & everything else gated or prompted for clarification. From there we scaled to full product with iteratively refined rules & a compacted schema model feeding a prompt that clarifies intent, routes to the right MCP tooling and lets the orchestration layer automate the rest.

4. Small teams can out-deliver large vendors if they wear the right hats. We had no product manager, no designer, and no AI specialists. Rather, we had a team of CDP subject matter experts that was willing to do work across every discipline, hustle and move at speed iterating fast with customers.

5. Be ready to evolve and hustle, especially while building any AI solution. We read books on building AI solutions that were outdated before we finished, went through different suggested frameworks/tooling, were recommended different vendors within GEICO—but in the end we did what worked best for our use cases keeping our customers in mind, picking the right models and making sure we continue to evolve as the AI tech story grows & unfolds in more ways around the globe.

Entire Engineering Team: Kriti Jha, Viswa Rajagopalan, Haitham Talouli, Alan Wong, Amy Tang, Jiawei Liu, Sachin Jhunjhunwala, Fan Yang.

Part 2 will cover the aggregation/OLAP query layer now in progress.

References

[1] Microsoft Engineering (ISE). “Ground Truth Curation for AI Systems.” https://devblogs.microsoft.com/ise/ground-truth-curation-for-ai-systems/

[20] AWS Well-Architected. “GENPERF01-BP01 Define a ground truth data set of prompts and responses.” https://docs.aws.amazon.com/wellarchitected/latest/generative-ai-lens/genperf01-bp01.html