HTTP Content Negotiation

Content negotiation is the mechanism by which a client and server agree on the best representation of a resource. The server selects from available variants based on client preferences expressed through HTTP headers, or presents alternatives for the client to choose from.

Table of Contents

Usage

A single resource at a given URL often has multiple representations. A news article exists in English and French. A product image is available as WebP, AVIF, and PNG. An API endpoint returns JSON or XML. The server needs a way to determine which variant to send.

Three negotiation patterns exist:

  • Proactive negotiation: the server picks the best representation using preferences the client sent in the request.
  • Reactive negotiation: the server lists available representations and the client picks one.
  • Request content negotiation: the server advertises preferences in a response, influencing how the client formats subsequent requests.

Proactive negotiation is the dominant pattern on the web. The client includes Accept, Accept-Encoding, and Accept-Language headers, and the server responds with the closest match. A Vary header in the response tells caches which request headers influenced the selection.

Proactive negotiation

The client states its preferences up front. The server inspects these headers and selects the representation best satisfying all constraints. No extra round-trip is needed.

The trade-off: the server is guessing at what the client wants, and sending those preferences on every request adds overhead and exposes a fingerprinting surface.

Accept

The Accept header lists preferred media types ranked by quality values. A browser navigation request sends a header similar to:

Accept: text/html, application/xhtml+xml, application/xml;q=0.9, image/avif, image/webp, */*;q=0.8

An API client requesting JSON with an XML fallback:

Accept: application/json, application/xml;q=0.9

An AI agent requesting Markdown to reduce token consumption:

Accept: text/markdown, text/html;q=0.9

AI agents and content negotiation

CDN providers convert HTML pages to Markdown on the fly when a request includes Accept: text/markdown. The text/markdown media type is a registered media type. Cloudflare's Markdown for Agents feature performs this conversion at the edge, reducing token counts by up to 80%. The response includes Content-Type: text/markdown and an x-markdown-tokens header estimating the token count. For error responses, requesting Accept: application/problem+json returns structured problem details instead of HTML error pages, reducing error payloads by over 98%.

When multiple types share the same quality value, the more specific type takes precedence. text/plain;format=flowed outranks text/plain, which outranks text/*, which outranks */*.

A quality value of 0 marks a type as not acceptable. When no quality value is specified, the default is 1.0.

Accept-Encoding

The Accept-Encoding header lists acceptable content codings. IANA maintains the registry of content codings. The most common values:

Coding Algorithm
gzip LZ77 with 32-bit CRC
deflate zlib-wrapped deflate
br Brotli
zstd Zstandard
dcb Dictionary-compressed Brotli
dcz Dictionary-compressed Zstandard
identity No encoding applied

Chromium-based browsers and Firefox send gzip, deflate, br, zstd over HTTPS. Over plain HTTP, Brotli and Zstandard are excluded.

Accept-Encoding: br, gzip;q=0.9, identity;q=0.1

An empty Accept-Encoding header signals the client wants no content coding applied. Omitting the header entirely means any coding is acceptable.

Accept-Language

The Accept-Language header specifies preferred natural languages using BCP 47 language tags.

Accept-Language: en-GB, en;q=0.8, de;q=0.5

The client prefers British English, accepts other English variants, and falls back to German. The server picks the closest available match.

Privacy

Sending a complete list of language preferences on every request creates a fingerprinting vector. A user agent without user control over language preferences must not send Accept-Language.

SEO and language negotiation

Google recommends using separate URLs with Hreflang annotations over Accept-Language negotiation. Googlebot and Bingbot both skip Accept-Language, relying on in-page signals such as lang attributes and Content-Language for language detection instead.

Accept-Charset

Accept-Charset specified character encoding preferences. All modern browsers now treat UTF-8 as the universal default, making this header obsolete and deprecated.

Client hints

Client Hints extend proactive negotiation with an opt-in model. The server sends Accept-CH to request specific hints, and the client includes them on subsequent same-origin requests over HTTPS.

Accept-CH: Sec-CH-UA, ECT, Device-Memory

Hints fall into three categories:

Unlike traditional Accept headers sent on every request, Client Hints require server opt-in and respect origin boundaries. This addresses the privacy and efficiency concerns inherent in proactive negotiation.

When responding based on hints, include the hint names in the Vary header so caches store separate entries per variant.

Reactive negotiation

In reactive negotiation, the server presents alternatives and the client makes the final choice. The server returns a 300 Multiple Choices response with a list of available representations, each identified by its own URI. The Location header points to the server's preferred option.

HTTP/1.1 300 Multiple Choices
Location: /article.en.html
Content-Type: text/html

<ul>
  <li><a href="/article.en.html">English</a></li>
  <li><a href="/article.fr.html">French</a></li>
  <li><a href="/article.de.html">German</a></li>
</ul>

A 406 Not Acceptable response signals proactive negotiation failed and no acceptable representation exists. In practice, servers rarely return 406. Most serve the default representation regardless of what the client requested.

The Vary header

The Vary response header serves two functions in content negotiation:

  1. Cache key expansion: Caches store separate responses for each unique combination of the listed header fields.
  2. Negotiation signal: Informs the client the response was selected based on specific request headers.
Vary: Accept, Accept-Encoding, Accept-Language

Vary: * signals unlimited variance. A cache stores a unique response per HTTP request. Only origin servers generate Vary: *.

When content is negotiated based on Client Hints, include the hint header names in Vary:

Vary: Accept-Encoding, Sec-CH-UA-Mobile, ECT

Example

A full proactive negotiation exchange. The client requests an HTML document in British English, compressed with Brotli.

Request

GET /report HTTP/1.1
Host: www.example.re
Accept: text/html, application/xhtml+xml, application/xml;q=0.9, */*;q=0.8
Accept-Language: en-GB, en;q=0.8
Accept-Encoding: br, gzip;q=0.9, zstd;q=0.8

Response

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Language: en
Content-Encoding: br
Vary: Accept-Encoding, Accept-Language
Content-Length: 4821

<!doctype html>...

The server selected HTML in English, compressed with Brotli. The Vary header tells caches to store separate entries for different Accept-Encoding and Accept-Language combinations.

Takeaway

Content negotiation enables clients and servers to agree on the best representation of a resource. Proactive negotiation through Accept, Accept-Encoding, and Accept-Language handles the majority of cases. Client Hints extend this model with privacy-respecting opt-in signals. Reactive negotiation through 300 Multiple Choices provides a fallback when the server cannot determine the best match. The Vary header ties negotiation into the Caching layer, ensuring caches store and serve the correct variants.

See also

Last updated: March 11, 2026