# Why AI Search Ignores Your Content — and How to Diagnose It

AI search systems don't match keywords — they decompose queries into themes, extract concepts, score content chunks against each theme, apply attention weighting, and select citations based on coverage completeness. Content that scores well for one theme but misses others will fail the overall retrieval threshold. Diagnosing this requires replicating the full pipeline for a specific query against a specific page.

AI Query Simulation in [QueryBurst](/content/site-root.html) replicates this 9-stage pipeline — [query fan-out](/content/use-cases/query-fan-out-tool/index.html), concept extraction, RAG scoring, attention scoring, and citation selection — with per-stage visibility into where content passes or fails.

## How It Works

### The 9-Stage Pipeline

The simulation replicates the core retrieval and generation pipeline used by modern AI systems:

```
1. QUERY FAN-OUT
   Your query → Multiple thematic queries
   Prioritized by importance (P1/P2/P3)

2. CONCEPT EXTRACTION
   Each theme → Key concepts AI focuses on
   Types: entities, intent, modifiers, context
   Weighted by importance

3. PAGE SCORING (RAG)
   How well your page covers each theme
   Traditional semantic search scoring
   Per-theme and overall page score

4. ATTENTION SCORING
   Concept-level weighted scoring
   Simulates what LLM "pays attention to"
   Identifies specific concept gaps

5. SITE-WIDE SEARCH
   Searches your entire website
   Two-phase retrieval (page → chunks)
   Finds best content across all pages

6. COVERAGE ANALYSIS
   Compares target page vs site best
   Identifies internal linking opportunities
   Shows where other pages beat yours

7. AI RESPONSE GENERATION
   Simulates actual AI response
   Shows which chunks get cited
   Tracks citations from your page

8. ITERATIVE REFINEMENT
   Identifies gaps in initial response
   Runs follow-up queries
   Generates refined response

9. RECOMMENDATIONS
   Actionable optimization suggestions
   Content gaps, internal links, enhancements
   Prioritized by impact
```

## Starting a Simulation

### Query Input

Enter a question a user might ask an AI assistant:

**Good queries:**

- "best golf clubs for beginners to improve my game"
- "how to choose ergonomic office chair for back pain"
- "what are the benefits of meditation for anxiety"

**Tips:**

- Use natural language (how users actually talk to AI)
- Include intent and context
- Think about what a complete answer needs

### Simulation Time

**Duration:** 30-60 seconds

- Shows real-time progress through stages
- Results are cached for instant re-access
- Can re-run to verify improvements

## Understanding Results

### Score Summary

Two key scores at the top:

**Page Score (RAG)**

- How well this specific page answers the query
- Traditional semantic search scoring
- 75+ = Strong coverage
- 60-74 = Good coverage
- 45-59 = Weak coverage
- <45 = Poor coverage

**Site Score**

- Best coverage available across your entire site
- Shows if other pages have better content
- Higher than page score = opportunity to link or consolidate

**Themes Count**

- Number of thematic queries generated
- Typically 5-10 themes
- More themes = more comprehensive query

**Actions Count**

- Number of optimization recommendations
- Prioritized by impact

### Thematic Fan-Out

**What it shows:** How AI decomposes your query into specific themes to research.

**Visual representation:**

- Query at top
- Branches out to themes below
- Size indicates importance
- Hover for full expanded query and reasoning

**Priority levels:**

| Priority | Visual | Meaning |
| --- | --- | --- |
| **P1** | Large, violet, glowing | Primary focus - must answer these |
| **P2** | Medium, gray | Supporting information |
| **P3** | Small, subtle | Contextual background |

**Example:** Query: "best latex mattress for back pain"

- P1: "Latex mattress recommendations for back pain relief"
- P1: "Latex vs memory foam for spinal support"
- P2: "Firmness levels for back pain sufferers"
- P2: "Certifications and materials quality"
- P3: "Price ranges and value comparison"

### Concept Extraction

**What it shows:** The specific concepts AI would focus on when reading your content.

**Concept types:**

| Type | Color | What It Is | Example |
| --- | --- | --- | --- |
| **Entity** | Cyan | Named things | "GOLS certification", "Dunlop latex" |
| **Intent** | Violet | User goals | "pain relief", "durability" |
| **Modifier** | Amber | Qualifiers | "organic", "best", "affordable" |
| **Context** | Gray | Background | "Australia", "2026" |

**Weight:**

- Shows how much attention AI gives each concept
- Higher weight = more important to address
- 60%+ weight concepts are critical

### Page Scoring (RAG)

**What it shows:** How well this specific page covers each theme using traditional semantic search.

**Priority breakdown:**

- Shows your average score for P1, P2, P3 themes separately
- P1 score is most important—must be strong to get cited

**Theme details (expandable):**

- Each theme with its score
- Best matching chunk from your page
- Heading context (where the chunk appears)

**Color coding:**

- Green (75%+) = Strong coverage
- Blue (60-74%) = Good coverage
- Amber (45-59%) = Weak coverage
- Red (<45%) = Poor/missing coverage

### Attention Scoring (Concept-Level)

**Advanced scoring** that simulates LLM attention mechanism.

**What's different from RAG:**

- Breaks themes into weighted concepts
- Scores each concept individually
- Calculates weighted overall score
- Shows specific concept gaps

### Site-Wide Coverage

**What it shows:** Whether other pages on your site have better content for each theme.

**Columns:**

- **Pri** - Theme priority (1/2/3)
- **Theme** - Theme name
- **Page** - This page's score
- **Best** - Best score from any page on site
- **Source** - Which page has the best content

### Retrieved Chunks

**What it shows:** The specific chunks that AI would retrieve and use to generate its response.

### AI Response Preview

**What it shows:** A simulated AI response using your content, exactly as [ChatGPT](/content/use-cases/optimize-website-chatgpt/index.html)/Perplexity would generate it.

### Recommendations

**What it shows:** Prioritized, actionable suggestions based on the simulation.

## Common Use Cases

### 1. Pre-Publication Check

**Goal:** Verify content will perform well in AI responses before publishing

### 2. Competitive Content Analysis

**Goal:** Understand why competitors get cited over you

### 3. Internal Linking Strategy

**Goal:** Optimize internal links to surface best content

### 4. Content Gap Discovery

**Goal:** Find missing topics/angles to create content for

### 5. Concept Coverage Audit

**Goal:** Ensure comprehensive topic coverage

## Best Practices

### Writing Good Queries

✅ **Do** use conversational language

✅ **Do** include user intent and context

✅ **Do** think comprehensively (what makes a complete answer?)

### Interpreting Scores

**Page Score Guidelines:**

- 80%+ = Excellent, highly competitive
- 70-79% = Strong, likely to be cited
- 60-69% = Good but improvable
- 50-59% = Weak, unlikely to be cited
- <50% = Poor, major gaps

### Optimization Workflow

1. **Run baseline** - Understand current state
2. **Prioritize P1** - Focus on primary themes first
3. **Address concept gaps** - Add high-weight missing concepts
4. **Check chunks** - Ensure best chunks are retrievable
5. **Re-run** - Verify improvements
6. **Iterate** - Keep refining until competitive

## Limitations

**What this simulates:**

- Core retrieval mechanisms
- Theme decomposition and concept extraction
- Citation selection logic
- Multi-turn refinement

**What it doesn't simulate:**

- Real-time web results (uses your crawled pages only)
- User engagement signals
- Personalization
- Platform-specific biases