Your website speaks HTML. Agents would rather read Markdown.

You optimized the page for Google. You made it responsive. You argued about the hero copy. Then an AI agent shows up and gets 500KB of document scaffolding for a few kilobytes of useful content.

That is the normal experience today: nav bars, cookie banners, script tags, tracking pixels, and somewhere in the middle, the thing the agent came to read.

We can do better. The web already has the mechanism.

Your layout is useful. Until it isn’t.

When a person visits a page, the browser turns all that machinery into a useful interface. CSS creates hierarchy. JavaScript adds behavior. Navigation helps someone move around.

An agent usually needs something simpler: the title, the content, the links, and enough metadata to understand what it found. Every extra token competes with the information it is actually trying to use.

The Vercel team put some numbers on this: a typical blog post weighs around 500KB with all the HTML, CSS, and JavaScript. The same content as Markdown? 2KB. That's a 99.6% reduction in payload size.

The fix is older than the problem

HTTP has supported content negotiation since the early days. The Accept header lets a client tell the server what format it wants. Browsers send Accept: text/html. Image tags send Accept: image/*. This is how the web has always worked.

The new part is clients asking for Markdown. A request can advertise that preference like this:

Accept: text/markdown, text/html, */*

That means: Markdown first, HTML if needed, anything else as a last resort. If your server respects the preference, it can return the same page as clean, structured Markdown instead of a browser document.

HTTP/1.1 200 OK
Content-Type: text/markdown; charset=utf-8
Vary: Accept

---
title: My Blog Post
date: 2026-03-15
author: Jane Smith
---

# My Blog Post

The actual content, with no wrapper markup.

No browser chrome. No scripts. No tracking. Just the content and the metadata needed to understand it.

What good content negotiation looks like

David Cramer wrote about how Sentry approaches this. A few things stood out.

Strip the browser chrome. Navigation, sidebars, JavaScript, cookie banners. None of it helps an agent. When you detect a Markdown request, serve only the content.

Restructure your link hierarchy. Index pages can become sitemaps. Instead of rendering a grid of cards with thumbnails, return a structured list of links with descriptions. Agents read partial responses (the first N lines) to decide if they need the rest. Put the most important information first.

Think about what agents do next. If a page requires authentication, don't just show a login form. Tell the agent that auth is required and point it to the API or CLI alternative. Agents behave differently when you tell them information exists versus making them discover it on their own.

The Vercel team takes a similar approach. Their implementation uses Next.js middleware to check the Accept header, then routes Markdown requests to handlers that convert their CMS content into clean Markdown. The key detail: code blocks keep their syntax highlighting markers, headings maintain hierarchy, and links stay functional. Structure matters, not just raw text.

And no, this is not llms.txt

You might have seen the llms.txt proposal floating around. It's a static file (like robots.txt) that describes your site for agents. It solves a different problem.

• robots.txt controls access. Which pages can be crawled.
• llms.txt provides a site overview. What your product does, where the docs live.
• Content negotiation controls format. What agents receive when they fetch a specific page.

They complement each other. llms.txt helps agents understand your site. Content negotiation helps them consume your pages efficiently.

Why bother now?

AI search products, coding agents, and custom research tools fetch pages every day. If your server ignores Accept: text/markdown, those clients have to extract the useful bits themselves. They may succeed. You are still making the job slower, noisier, and less reliable than it needs to be.

Will a Markdown response guarantee a citation? Of course not. It does make your content cheaper to retrieve and easier to parse, which is a much more defensible goal than trying to reverse-engineer an “AI SEO” trick.

The implementation is conceptually straightforward: check the Accept header, render the page content without its browser shell, add useful frontmatter, and return the right Content-Type. Production details like caching and routing still matter. The core idea does not need to be complicated.

Test where you stand

This is what we built AgentReady.dev to test. Enter a URL and we request it as both HTML and Markdown, compare the responses, measure the bloat, and score how well the site serves this new kind of reader.

It is free, there is no signup, and it takes about 30 seconds. Run an audit and see what your server actually sends back.