Replicating Exa Highlights (With a Potato And Some String)

Can we replicate Exa's Highlights demo with off-the-shelf embeddings and Python? Can we create something that's even more token efficient? Let's find out.

David McSweeney

May 12, 2026

Try the full interactive pipeline demo here (opens in a new tab)

Google recently announced Grounding With Exa Web Search for Gemini models.

This new grounding option, which is currently available in private preview on Vertex, uses Exa’s “Highlights” to extract query relevant text from a web page (or pages), leading to a significant reduction in input tokens.

Highlights is cool. And it looks like magic…

…but I was always the kid that was trying to peek up the magician’s sleeve. And I was skeptical about it being a single model doing all the work.

My Original Hypothesis (Based On Their Demo Video)

Watching the demo closely, and noticing that the highlights shown were all partial sentences/clauses, I formed an initial hypothesis about what Exa's "trained model" actually is. They claim their API returns highlights in <100ms. But inference over a 10K token document takes 50-500ms minimum.

So I figured they were doing something like:

• pre-computed sentence embeddings at index time,
• approximate nearest-neighbour lookup at query time,
• a lightweight re-ranker (cross-encoder) to score the top candidates,
• a clause boundary detector to trim sentences down to informative fragments,
• and some form of query classifier for routing.

And I set out to build V1 to emulate it.

V1: matching the Exa demo

I built the first version in a day or so, trying to replicate what the demo appeared to show, which again, was sub-sentence, clause-level highlights. My pipeline was:

1. Fetch and clean — Firecrawl grabs the page as markdown, stripping navigation, headers, images
2. Split into sentences — declarative prose only, filtered for length and quality
3. Generate hypothetical answer (Split HyDE) — a cheap LLM generates a hypothetical answer (doesn't have to be right, just use the right words), split into individual sentences, each embedded separately
4. Embed everything — batch embed all document sentences + HyDE vectors
5. Score and rank — cosine similarity (max-sim across HyDE vectors) + n-gram keyword boost
6. Extract clauses — isolate the informative complement clause from each top sentence

On SimpleQA (feeding the extracted highlights to an LLM for answer generation), this version scored 75% accuracy with ~397 tokens, beating full-page retrieval (65% at ~878 tokens) by 10 percentage points. I used a fast, cheap model (Gemini 2.5 Flash Lite) at every stage.

The clause extraction was working pretty well. Exa's demo showed inline sub-sentence fragments, so I built regex-based clause boundary detection: find the declarative verb, extract the complement. "Exa is a custom search engine built for AIs" → "a custom search engine built for AIs."

I was ready to publish. "Here's how Exa probably does it, here's our replication, here's the benchmark."

Then I tested their actual API...

V2: what I built after seeing the truth

The API results revealed that Exa isn't doing sub-sentence extraction at all. Their demo runs against their own docs page, where section boundaries happen to look like clause-level highlights. On real prose, they often return section-level blobs.

This changed the problem. Instead of trying to match their demo, I could build what the demo claims to do. The pipeline evolved:

1. Fetch and clean — now retaining heading structure for analysis
2. Classify retrieval strategy — a single lightweight LLM call reads the page's heading outline and decides the optimal extraction approach (this is new — and probably what Exa's "trained model" aspires to)
3. Early exit or full pipeline — the classifier determines what happens next:
   • HEADINGS_DIRECT → return the heading text as the answer. No embeddings, no scoring. Done.
   • SECTION_PROSE → retrieve the target section's content directly. No embeddings needed. Done.
   • FULL_EXTRACTION / FULL_EXTRACTION_TRIPLES → run the full V1 pipeline (split, HyDE, embed, score, rank), optionally with triple extraction on top

The output is scored highlights in three formats (sentences, clauses, triples) — not answers. Answer generation is the consumer's job. We generate answers in the demo to prove the highlights are groundable, but that's not part of the extraction pipeline itself.

The key additions: adaptive routing (the classifier) and structured compression (semantic triples). Each emerged from testing against real content and finding where the V1 pipeline fell short.

What Exa's API actually returns

I signed up, got an API key, and ran the same queries against their highlights endpoint. The results weren't what I expected.

QueryBurst Highlights

Verdict: Ten scored evaluative judgments pulled from throughout the review vs. one paragraph of plot summary.

Exa Highlights

Inasmuch as 2015's new trilogy opener, The Force Awakens, modeled itself (heavily) on the original Star Wars film, the second installment, The Last Jedi, is the Empire of the current batch.

Verdict: Both get it right. Exa returns the intro paragraph (correct for this query type). Our classifier targets the "Founding" section specifically and gets "December 2015" — more precise.

The adaptive classifier: doing what Exa's "trained model" claims to do

The concept behind Exa's approach is sound. Not all queries need the same extraction method. A listicle query should grab headings. A factoid should find the right paragraph. An opinion question needs to sweep the whole article. Their "trained model" is presumably trying to learn this routing. I built it explicitly: a single lightweight LLM call (~1.4s) that reads the page's heading outline and decides what to do.

In production, this is a textbook distillation target:

1. Bootstrap labels — the LLM classifier generates its own training data. Every query it processes is a labelled example: (query, heading outline) → strategy.
2. Train a small model — a fine-tuned BERT, a lightweight encoder, or even logistic regression on heading features (count, depth, question-mark presence, list patterns, heading-to-content ratio). The decision boundary isn't complex.
3. Swap in — the trained model replaces the LLM call. Latency drops from ~1.4s to <10ms. Cost drops to zero per query. The rest of the pipeline stays identical.

The above steps are almost certainly the path Exa took for their "trained model". The classifier concept is the same; the difference is what happens after routing.