HTTP Content Negotiation

Content negotiation is the process whereby the client and server choose one of perhaps several representations of the target resource that exist on the server. The choice is intended to be the one best suited for the client, such as an English version of a document versus a French version.

Usage

When a client requests a resource using a URL, it is usually the URL for the overall resource, which includes all of its representations. Each representation is a different version of the resource and behind the scenes, each of these representations has its own URL. The server is responsible for selecting which specific version to send and this process is called content negotiation.

Type of negotiation

There are two approaches to content negotiation, namely server-driven and agent-driven. Server-driven negotiation is also known as proactive negotiation, and it is the most common method. It is initiated by the client and is accomplished by including certain HTTP headers with each HTTP request.

Agent-driven negotiation is also known as reactive negotiation, and it is initiated by the server. Negotiation of this type begins with the server including particular HTTP response status codes, and sometimes a tailored message body as part of the HTTP response.

Server-side negotiation request headers

There are a limited number of HTTP headers that a client can include to ask for a specific representation of a resource. These are Accept, Accept-Charset (no longer used), Accept-Encoding and Accept-Language. When a client includes one or more of these HTTP headers, it is not a guarantee that the server will be able to generate a representation that satisfies all of the constraints. Rather, it will use all of the information it has to best select or generate the most suitable one.

For example, if the client requests a document in British English (en-GB), but the server only has versions available in US English (en-US), French (fr), and German (de), then the server will pick US English because it is the closest. When the server responds, it will include the Vary header to indicate which HTTP request headers were used during the negotiation phase.

Alternatively, the server may respond with the 406 Not Acceptable or 415 Unsupported Media Type HTTP status code. This implies that the server is not able or not willing to guess at or generate the best substitute and instead, directs the client to include a valid choice in a new HTTP request.

Accept

The Accept HTTP request header includes a list of MIME-encoding options that the client is willing to accept. It is formatted as a comma-delimited list that includes each MIME type, as well as a quality factor. The quality factor (q) is an indicator to inform as to the relative degree of preference between the various options. A type with no quality factor is assumed to be 1.0, or 100%, and most preferable. If two types have the same quality factor then the more specific one (e.g. en-GB rather than en) will be selected.

For example, Chrome uses the following Accept header by default:

text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8

The final option in the list, */*, is a wildcard that means all types are accepted.

Accept-Charset

Accept-Charset was used by clients to specify which character sets they are willing or prefer to accept.

Accept-Encoding

Accept-Encoding is available for clients to ask for specific content-codings in the response. The directives can include Compression algorithms such as gzip, compress, deflate, deflate-raw and br. The identity directive is always available, even if it is not specified, and is assumed to be at the lowest priority. The asterisk (*) is available as the wildcard character, allowing for any encoding that was not listed in the HTTP header.

Accept-Encoding: br, deflate;q=0.8

In this example, br is the preferred compression algorithm but the client will accept deflate, and then uncompressed (identity) as a last resort.

Accept-Language

The Accept-Language HTTP request header is sent by the client to indicate the preference of language. It consists of an ordered list of language codes and a quality factor. The default is normally set by the client’s browser, however, this is something that can normally be changed on the entry page to a website. Once the language is selected, it will be used for the remainder of the interaction with the client. HTTP requests that do not include the Accept-Language header are implicitly stating that the client is willing to accept any language.

Accept-Language: en-GB, en;q=0.8, de, fr;q=0.7

In this example, the client informs the browser that it prefers to receive the resource in British English, and secondarily, that all other variations of English are accepted. If these are not available then with a lower preference, the client will accept resources in German or French.

Client hints

Two directives are part of the Client Hints feature, which is still an experimental part of HTTP. The two directives are Accept-CH and Accept-CH-Lifetime and the latter has now been deprecated.

Accept-CH

Accept-CH is sent by the server to inform the client that it supports hints and which directives it is willing to accept. The possible values are:

• Device-Memory: This indicates the approximate amount of RAM the device has available, measured in gigabytes.

• Viewport-Width: This indicates the size of the client’s layout viewport, measured in CSS pixels. Note: for many devices, the viewport is smaller than the advertised resolution.

• Width: This indicates the width of the resource in physical pixels.

When the server receives hints from the client, it may respond with the Vary HTTP response header to indicate which Client Hints are accepted.

Accept-CH-Lifetime

Accept-CH-Lifetime works in conjunction with Accept-CH, where a number is included to indicate persistence for a specific number of seconds.

Server-side negotiation response headers

HTTP response headers are part of the content negation process, both for server-driven and agent-driven negotiation. In the more common server-driven approach, the Vary header is sent by the server to indicate which HTTP request headers were used.

Vary

The Vary header is sent by the server so that a cache will understand what decisions the server made. This is required so that the cache can recreate the steps and deliver the appropriate content. Without being able to reproduce the message body, Caching the HTTP response is not possible.

When a cache receives a message that includes the Vary header, it must ensure that all of the fields specified in the header match that of the cached HTTP request.

The asterisk (*) is used to inform the cache that additional information, not included in the HTTP Vary header, is used in the process. In this case, Caching is not possible.

Vary: Accept-Encoding

In this example, the server informs the cache that the Accept-Encoding header was used. As such, the cache will ensure that the correct compression is indeed cached before serving the content.

Agent-side negotiation

In agent-side negotiation, when the server receives a HTTP request that can be answered in multiple ways, it returns a selection page and includes the 300 Multiple Choices status code. The client is responsible for choosing the most suitable resource among the options that are presented.

Example

In this example of server-driven content negotiation, the client requests an HTML resource. It wants the British English version but is willing to accept any English representation. The client asks that it be compressed, preferably using the br algorithm, although it supports deflate as well. The server responds with a representation of the resource that satisfies all of these constraints.

Request

GET /news.html HTTP/1.1
Host: www.example.re
Accept: text/html
Accept-Language: en-GB, en
Accept-Encoding: br, deflate;q=0.8

Response

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 500
Content-Language: en
Content-Encoding: br
Vary: Content-Encoding

<HTML document is included in the message body>

Takeaway

HTTP content negotiation is a process whereby a specific representation or version of a resource can be retrieved out of several possible options. The negotiation can be driven by the client or the server. Clients can use a specific set of HTTP headers to make requests about characteristics such as the language and can indicate a preference using a weight called the quality factor. Support for Caching is available by using the Vary header.