Serving Markdown Instead of HTML to AI — Taking zench-aine.io to Level 3 (Markdown Negotiation)

Introduction

In part one we explained Cloudflare's Agent Readiness Score ("Lighthouse for AI agents"), and in part two we actually fixed zench-aine.io to go from Level 1 to Level 2. This follow-up tackles the next step: Level 2 → Level 3 "Agent-Readable." The key is Markdown content negotiation.

As noted in part two, zench-aine.io is built on Next.js 15 (App Router) with Velite, an MDX-based content library. Since this article is implementation-heavy, that context comes first.

Getting the Fix from the Scanner's "Copy Prompt"

A genuinely helpful part of the Agent Readiness Score is the "Copy prompt" button on each failing check. Press it and the fix instructions are copied to your clipboard, ready to paste into a coding agent like Cursor or Claude Code.

Pressing "Copy prompt" on the Markdown Negotiation check gives you this:

Goal: Return HTML responses as markdown when agents request it
 
Issue: Site does not support Markdown for Agents
 
Fix: Enable Markdown for Agents so requests with Accept: text/markdown
return a markdown version of your HTML response while HTML stays the
default for browsers. Confirm the response uses Content-Type:
text/markdown (and x-markdown-tokens if available).
 
Skill: https://isitagentready.com/.well-known/agent-skills/markdown-negotiation/SKILL.md
Docs: https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/

Goal, Issue, and Fix are stated concisely, plus links to an Agent Skills-format SKILL.md (an agent-facing implementation guide) and the official docs. That SKILL.md spells out the requirements — "return Content-Type: text/markdown for Accept: text/markdown," "keep HTML the default for browsers," "add an x-markdown-tokens header if possible" — and we implemented exactly to it.

What Is Markdown Negotiation, and Why Does It Help?

Markdown content negotiation is an HTTP mechanism where the server returns clean Markdown instead of verbose HTML, but only when the client sends Accept: text/markdown. Ordinary browsers (requesting HTML) still get HTML, so the human experience is unchanged.

For AI agents, there are two big wins.

1. Major token reduction — navigation, ads, and decorative HTML disappear, leaving body-only Markdown. In Cloudflare's example, 12,345 HTML tokens become 725 Markdown tokens (~94% less). Agents read more articles with less context and lower cost.
2. Fewer misreads — clean, structured Markdown is easier to parse, with no confusion between content and chrome, raising comprehension and citation accuracy.

For a media site aiming to be read and cited accurately and efficiently by AI, this is an essential move. And in the Agent Readiness Score, it is the gate to Level 3 "Agent-Readable."

Why Skip DNS-AID?

Among the same failing checks, we deliberately skipped DNS for AI Discovery (DNS-AID). Here's the reasoning.

DNS-AID publishes agent or MCP-server endpoints in DNS (SVCB / TLSA / DNSSEC) so other agents can discover and verify them without a third-party registry. In other words, it's for organizations that provide agents or services.

zench-aine.io is a media site that serves articles; it has no agent/MCP endpoint to publish in DNS. The IETF/Cloudflare guidance itself says content/media sites without agent endpoints don't need DNS-AID. On top of that, it's still an Internet-Draft (not a formal standard) and carries operational overhead like DNSSEC. Implementing it just for the score would only add settings nobody uses — so skipping it was the right call.

Implementing Markdown Negotiation in Next.js

zench-aine.io is not behind Cloudflare's proxy (it's direct Cloud Run), so the dashboard "Markdown for Agents" feature isn't available. We self-implement in Next.js. The approach: detect Accept: text/markdown in middleware and rewrite to a Route Handler that returns Markdown.

First, the middleware routing:

// middleware.ts
// True only when text/markdown is accepted with q > 0 (excludes an explicit q=0 refusal)
function acceptsMarkdown(accept: string): boolean {
  for (const part of accept.split(',')) {
    const tokens = part.trim().toLowerCase().split(';').map((s) => s.trim())
    if (tokens[0] !== 'text/markdown') continue
    const qToken = tokens.find((t) => t.startsWith('q='))
    const q = qToken ? Number.parseFloat(qToken.slice(2)) : 1
    if (!Number.isNaN(q) && q > 0) return true
  }
  return false
}
 
export default function middleware(request: NextRequest) {
  const accept = request.headers.get('accept') || ''
  if (acceptsMarkdown(accept)) {
    const url = request.nextUrl.clone()
    url.pathname = `/api/markdown${url.pathname === '/' ? '' : url.pathname}`
    return NextResponse.rewrite(url)
  }
  return intlMiddleware(request)  // otherwise, next-intl routing
}

A naive substring match on text/markdown would also catch Accept: text/markdown;q=0 — an explicit "I do not want Markdown." Checking the q-value and only rewriting when q > 0 is the correct HTTP content negotiation.

The destination Route Handler returns Markdown for the homepage, articles, and the index. For article bodies, we add raw: s.raw() to the Velite schema to get the raw Markdown and return it with frontmatter. It also returns 404 for unknown pages and over-long paths like /media/{slug}/extra, matching the HTML side so the Markdown endpoint isn't more permissive.

// app/api/markdown/[[...path]]/route.ts (excerpt)
function markdownResponse(body: string) {
  return new Response(body, {
    headers: {
      'Content-Type': 'text/markdown; charset=utf-8',
      'x-markdown-tokens': String(Math.ceil(body.length / 4)), // rough token estimate
      'Vary': 'Accept',
    },
  })
}

The key is to follow the SKILL.md requirements: return Content-Type: text/markdown and x-markdown-tokens, and set Vary: Accept so caches split correctly. Browsers (Accept: text/html) still get the original HTML, so the human experience doesn't change at all.

Locally, curl -H "Accept: text/markdown" https://zench-aine.io/ returned Markdown while a normal browser request returned HTML — exactly as intended.

What Did the Re-Scan Show? Level 3

After deploying to production and re-scanning, the Content category became 1/1 (Markdown Negotiation passes), and the score rose 36 → 43.

The scan API's level reached Level 3 "Agent-Readable," exactly as predicted in part two.

The gate to the next Level 4 "Agent-Integrated" is precisely MCP, A2A, Agent Skills, and API catalog — the service-provider items we've consistently called unnecessary for a content site. For zench-aine.io, Level 3 "Agent-Readable" is effectively the goal for now.

FAQ

Q. Does returning Markdown change what humans see?

A. No. Markdown is returned only to agents that explicitly send Accept: text/markdown; browsers (Accept: text/html) still get HTML. With Vary: Accept separating the cache, the human experience is entirely unaffected.

Q. Why not use Cloudflare's feature?

A. Because zench-aine.io isn't behind Cloudflare's proxy (it's direct Cloud Run). Behind the proxy you could enable it with a dashboard toggle, but our setup rules that out, so we self-implemented in Next.js.

Q. Isn't llms.txt enough, making Markdown negotiation redundant?

A. They play different roles. llms.txt is a "table of contents for the whole site"; Markdown negotiation returns "each page's body as Markdown." The former helps discovery, the latter helps efficient reading — they complement each other.

Conclusion

Markdown content negotiation serves clean Markdown to agents sending Accept: text/markdown, cutting tokens sharply while improving comprehension and citation accuracy. Starting from the scanner's "Copy prompt," we self-implemented it in Next.js and took zench-aine.io from score 36 to 43, reaching Level 3 "Agent-Readable."

Meanwhile we deliberately skipped DNS-AID and the Level 4 MCP/Agent Skills items as service-provider concerns. Don't chase the score — identify the checks that fit your site's nature and act on those. That stance, consistent across the series, is what pays off most.

Together with part one (the explainer and analysis) and part two (the Level 1→2 hands-on), we hope this serves as a map for the agentic-web standards. At ZenChAIne, we share efforts like these with the hands-on detail intact.

References

• Markdown for Agents (Cloudflare docs)
• Markdown Negotiation SKILL.md (isitagentready)
• isitagentready.com (Agent Readiness Score)
• DNS for AI Discovery (IETF draft)
• Agent Skills