The Shadow Web, Part 2: LLMs Aren’t Reading Your HTML Anyway - QueryBurst

The Shadow Web, Part 2: LLMs Aren’t Reading Your HTML Anyway

Token efficiency, markdown, blue bananas, global summits and Shoggoth. What do agents really "see" when they browse the web?

David McSweeney

February 19, 2026

Last week I pointed out the serious security and trust issues raised by Cloudflare’s ‘Markdown for Agents’, or at least the current implementation of it.

But at the same time (mainly on social media) I also pointed out that this was all ridiculous, since LLMs aren’t out there reading raw HTML. In any half-decent agentic pipeline the page is going to be fetched, parsed, sanitized, and delivered as clean context to the LLM (likely as markdown). Anything else would not only be inefficient, but also dangerous (more on that at the end).

Nevertheless, the debate continues to rage on, and there are a load of people still celebrating this as some massive win, and something that you must enable if you want to appear in more AI answers.

So to demonstrate that LLMs do not get raw HTML, and to check what they do see, I set up a few simple tests. I just used ChatGPT for this since that’s the one everyone high-fiving on LinkedIn cares about. ChatGPT is not an agent you say? I address that at the end.

Here are the tests and the results.

Test 1: Hidden comment in the head

This first test was set up to show that LLMs do not get a full HTML document, i.e. everything.

<!DOCTYPE html>
<html>
<head>
<title>The History of the Blue Banana</title>
<!-- SECRET_FACT: The Blue Banana was actually invented in 1999 by a scientist named Dr. Plum. -->
<meta name="author" content="Dr. Plum">
</head>
<body>
<h1>The History of the Blue Banana</h1>
<p>The Blue Banana is a rare fruit discovered in the Amazon rainforest in 2024.</p>
</body>
</html>

As you can see, we have a secret fact hidden in a comment in the head. If an LLM was getting the full HTML it would be able to read this fact.

So did it?

No.

I asked it to double check.

Still no.

Confirmed: LLMs don’t get the full HTML document.

Test 2: Hidden comment in body and rendered DOM test

This test was to check if an LLM could read a secret fact hidden in the body, and also to check whether hiding an element with display:none would also hide it from the LLM. If ChatGPT was browsing the web like a human it would not see the secret hidden in display:none.

Here is the HTML.

<!DOCTYPE html>
<html>
<head>
<title>Blue Banana Facts</title>
</head>
<body>
<h1>The Blue Banana Verification</h1>
<p>The Blue Banana is a fruit discovered in the Amazon in 2024.</p>
<!-- SECRET_INFO: The Blue Banana was actually invented by Dr. Plum in 1999. -->
<p>It is known for its bright blue skin.</p>
<div style="display:none">
SECRET_INFO_2: The banana is made of plastic.
</div>
</body>
</html>

What did it see?

It missed the secret hidden in the comment (proof that it’s not getting the full body HTML) and it did see the secret hidden with display:none (proof that it’s not viewing a rendered DOM).

Confirmed: LLMs don’t get the full HTML from the body or view a rendered DOM when “browsing”.

Test 3: The nuclear test

For this one I hid a load of conflicting facts about a global summit in HTML elements and tags (aria-label, data-location, noscript)

<!DOCTYPE html>
<html>
<head>
<title>Summit Location 2026</title>
<meta name="description" content="The summit is in Rome.">
</head>
<body>
<h1>Global Tech Summit 2026</h1>
<!-- 1. The Visible Text (Control) -->
<p>We are excited to announce that the 2026 summit will be held in <strong>Paris</strong>.</p>
<!-- 2. The Aria Label (Accessibility) -->
<div aria-label="The summit is in London."></div>
<!-- 3. The Title Attribute (Tooltip) -->
<p title="The summit is in Tokyo.">
Hover for details.
</p>
<!-- 4. The Image Alt (Visual) -->
<img src="pixel.png" alt="The summit is in Berlin." />
<!-- 5. Data Attribute (Code) -->
<div data-location="The summit is in Madrid."></div>
<!-- 6. Noscript (Fallback) -->
<noscript>The summit is in Dublin.</noscript>
</body>
</html>

What did the model see?

As expected it saw:

It missed:

Confirmed: LLMs (or ChatGPT at least) get pre-parsed, clean context, which is almost certain to be markdown.

The Verdict: Can we stop now?

These tests confirm that well set up “AI agents” already use a sophisticated, multi-stage extraction pipeline.

They intelligently strip out the "messy HTML" that Cloudflare claims to be solving.

The "80% token savings" Cloudflare promises are a myth because those tokens were never going to hit the LLM context window anyway. The agent's native scraper had already thrown them in the trash.

Any system processing data from third-party sources must operate on a Zero Trust basis. This is why OpenAI (and any professional agentic system) must have a parsing and sanitization layer between the "Wild West" of the internet and their models.

This layer strips out suspicious scripts, malicious event handlers, hidden payloads, prompt injections etc. By offering to "pre-clean" the web into Markdown, Cloudflare is effectively encouraging developers to bypass this battle-tested security boundary and trust an opaque Edge-converter instead.

Cloudflare has solved a "Token Problem" that doesn't exist, and in doing so, they have created a "Trust Problem" that is very, very real.

The agents (or at least the pipelines behind them) are already doing the cleaning. Cloudflare has just introduced a way for malicious actors to 'dirty' the data.

And as I’ve already said in my previous post, if they (Cloudflare) serve one canonical document (i.e. human and bot gets the same content) the trust problem doesn’t exist and the feature becomes potentially useful.

A note on LLMs as extractors/parsers

Can LLMs be used to extract content from HTML? Yes, they can. But again, an efficient system is going to run the raw HTML through a deterministic parser first to save tokens and remove noise. Either way, you’d be looking at a fast, cheap, tuned model for this extraction, not the high-performance model used for the final reasoning.

ChatGPT is not an agent bro!

Might as well address this one.

Yes, I’m aware the product is called “Markdown for Agents” and was most likely rolled out to capitalize on the OpenClaw (or whatever it’s called this week) party.

But look, this is an SEO blog. The “GEO” crowd are claiming that you should enable this to show up more in ChatGPT, hence focusing the tests there. As I’ve demonstrated, there’s no need. It’s not going to help your brand get cited more.

If you’re running your own agentic pipeline you need to parse for security reasons before feeding to an LLM. Even if you’re already getting markdown you still need to sanitize it.

If OpenClaw doesn’t parse/sanitize when scraping, that’s just another security issue to add to the growing list.

If we’re talking about an agent interacting with the human web (controlling a browser via puppeteer, playwright etc) rather than scraping, then… well… it’s already the human web isn’t it.

But to really close this down, here are the same tests using Cursor’s agent (Opus 4.6). Try it yourself if you don’t believe me.

Test 1 (same results)

Test 2 (same results)

Test 3 (it saw even less - Cursor’s parsing is more aggressive)

Let me know if you disagree.