What exactly is Content Negotiation? According to Wikipedia it’s “a mechanism defined in the HTTP specification that makes it possible to serve different versions of a document at the same URI, so that user agents can specify which version fit their capabilities the best”.
photo by JD Hancock
So, it serves two purposes: (1) making it possible to have different versions of the same response, and (2) letting clients specify which version they want to receive. Usually, this technique is applied when there are several types of user agents consuming the same HTTP resource but, because they have different rendering capabilities, they might ask for different content types.
With APIs, this approach is followed whenever you’re able to handle different request and response formats, e.g. XML and JSON, and you’d like to offer the client the ability to choose which format to consume. The technique gained traction when JSON started having wide adoption and API providers didn’t want to break compatibility with existing consumers. Instead of replacing whatever they had (usually XML) with JSON, they offered Content Negotiation, letting clients decide which format to handle.
There are several ways to provide Content Negotiation, from capturing GET parameters to fully using the HTTP protocol headers as they were intended.
Capture GET parameters
Most probably, the easiest way to offer Content Negotiation is to capture one or several HTTP GET parameters and changing the input filtering and the output format accordingly. You’ll obviously have to document or announce which formats you’re offering and how they are activated, but there’s not much else to it. Here’s an example call that would receive some content in XML format and the corresponding response:
Call: GET /user/123?output=xml Output: <?xml version="1.0" encoding="UTF-8"?> <user id="123"> <firstName>First Name</firstName> <lastName>Last Name</lastName> </user>
Now, the same call can be made so that consumers receive JSON instead of XML:
Call:
GET /user/123?output=json
Output:
{
"id": "123",
"firstName": "First Name",
"lastName": "Last Name"
}
Another example is specifying both input and output formats using the same technique:
Call:
POST /user?input=json&output=json
Input:
{
"firstName": "First Name",
"lastName": "Last Name"
}
Output:
{
"id": "456"
}
This is by far the easiest way to implement Content Negotiation, but there are other ways to do it.
Interpret resource extension
Since every resource can have an extension like regular files do, you can interpret that extension and act accordingly. The concept is the same as previously defined but might require more interpreting on the server side. On the other hand, this approach lets you define different controllers that will be called depending on the resource extension, making the solution easier to develop and maintain. Here’s an example of a call using the technique:
Call:
GET /user/123.json
Output:
{
"id": "123",
"firstName": "First Name",
"lastName": "Last Name"
}
One thing to notice immediately is that you lose the ability to specify different input and output formats. While this is highly improbable to happen, if it’s a requirement you have you can’t choose this approach.
Use the right HTTP header
The best approach is to follow the standards. In this case, the HTTP protocol (RFC 2616, section 12) clearly defines how content should be negotiated between server and client. The protocol defines two different kinds of Content Negotiation:
- Agent-driven Negotiation: whenever the decision about content type or format is left to the client. The client makes a first call and obtains a resource which indicates possible alternative formats. A subsequent call is needed in order to obtain the intended resource representation.
- Server-driven Negotiation: whenever the server decides which content to deliver based on several factors obtained from the client call. In fact, all previous techniques are considered server-driven, but what’s interesting is a particular way to hint about which content types are supported: Accept headers.
Content Negotiation sequence diagram
The way these headers work is quite easy to understand. The client request will include a header specifying which formats are considered accepted. The server then chooses from those formats the one that best suits the purpose of the call. If only one format is specified by the client, the server responds using that format, if available. Here’s an example request:
Call:
HEADER Accept: application/json
GET /user/123
Output:
{
"id": "123",
"firstName": "First Name",
"lastName": "Last Name"
}
There are many other Accept header types that might hint the server about what types of information the consumer would prefer. The most important are related with data compression and localization:
- Accept-Encoding: let’s you specify which types of data encoding you’d like to receive. This is very useful because it hints the server that you can consume compressed data, e.g. sending “gzip” as the header content would let the server know that you can processed zipped content.
- Accept-language: by specifying which language you’d like to accept you can hint the server about localization options, such as currency or other specific international symbols.
Offer a reasonable default
If you used to support XML and now you’d also like to support JSON, offer XML as your standard so that you don’t break any existing consumers. Otherwise, offer whichever format is the most widely used by the majority of your consumers.
Content Negotiation, if used properly, can dramatically improve the UX of your API. You’ll get much closer to the way clients expect to consume your data making their lives easier and, in the end, making your final customers happier.
Please join in the discussion on this article over on Hacker News and reddit.
Nice post, I wrote something in similar context under http://www.dzone.com/links/implementing_api_faccedilade_with_wso2_api_manage.html .
Although it’s not exactly on the same topic, your article is very interesting, specially for someone not familiar with WSO2!
I also wrote something on the topic of Facades, for a less knowledgeable audience here: http://blog.cloudwork.com/using-facades-decouple-api/
Take a look and let me know what you think.
You should not ever need to provide an input parameter. The Content-Type header is required when sending a payload and will provide the media type.
Having clients construct a URI with a format=x is not really content negotiation because there isn’t really any negotiation going on. All that is happening is that you are providing multiple resources that expose different media types. This is necessary when doing agent driven negotiation, but in itself, it is not negotiation.
When discussing conneg it is important to consider caching issues and it is worth explaining how different options can be presented to a client to enable agent driven negotiation.
I disagree. In the context of Web APIs it’s important to show different options because some APIs are implementing them.
For example, Highrise API uses resource extensions as a mechanism to negotiate different formats (https://github.com/37signals/highrise-api#alternative-formats). They support XML as a standard but they can respond with VCF on some resources related with contacts.
Strictly speaking, HTTP Content Negotiation doesn’t involve GET parameters or resource extensions. As I wrote on the article “the best approach is to follow the standards”.
— For future readers —
For what it’s worth, you’re missing Darrel’s point. The first 2 ways to “provide Content Negotiation” – query and “extension” – are not about negotiation. If you have a static web server, you request a file with extension .json .png etc. No negotiation happens. Same with the query param. You request “http://example.com/x?format=json” – that’s the URI of the “document” – the atomic resource URI. The document you requested it’s not “http://example.com/x”
Negotiation is a give&take not all-or-nothing process. And that only happens for “the 3rd way”. The client can say “Accept: application/json, application/xml” and the server can reply with either of the two, or with text/html, and in all 3 situations the negotiation succeeded, or with 406 Not Acceptable – negotiation failed.
I appreciate your comment and clarification, but I didn’t miss Darrel’s point — I even agreed with him that the first two techniques were presented because many APIs rely on them to provide content negotiation.
Now, replying to you, if you read RFC 2616 §12.1 (Server-driven Negotiation) carefully, you’ll notice the following phrase:
“However, an origin server is not limited to these dimensions (request-header fields) and MAY vary the response based on *any aspect of the request*, including information outside the request-header fields or within extension header fields not defined by this specification.” (bolds and contextual information added by me)
Also, by reading RFC 2396 §1.1, you’d know that a “resource is the conceptual mapping to an entity or set of entities, not necessarily the entity which corresponds to that mapping” and by reading RFC 2295 (Transparent Content Negotiation in HTTP) §4.6 (Retrieving a variant by hand) you’d know that the “user agent can use this list (of variants) to make available a menu of all variants and their characteristics to the user.”
So, negotiation can assume many different forms, including one where the consumer directly requests a variant of the resource obtained from a previously published list (the API documentation, in our case).
I stick to what I wrote before, and please don’t assume that nobody reads the RFCs, albeit I reckon it is only a few.
FWIW I still think you’re missing the point – content negotiation is when you request X twice, and one time you can get JSON, one time you can get XML. Not when you request X1 and get JSON, and then you request X2 and get XML. It’s irrelevant that the resources map to the same entity, it is still resource*S*
But let’s leave it to each and every potential reader to make up their mind.
You say that content negotiation is “not when you request X1 and get JSON, and then you request X2 and get XML” — not according to RFC 2295 §4.6 (Retrieving a variant by hand), where the consumer requests different resources in order to receive different content types.
It seems that we agree on something, though: only a few people really read RFCs these days.
Anyway, thanks for the interesting discussion.
Great Post, Bruno!
On motivating the need/usefulness to support multiple data-formats (a.k.a. resource representations)…but should make a stronger point in defence of XML (not only “backward compatibility”).
If the client is not JavaScript, in principle, odds should favour XML over JSON (since its a more complete and rich technology ecosystem — e.g. more expressive syntax, namespaces, XSLT transformation, XPATH selector, better and more libraries, etc.).
However, because most apps and business models build these days are really around webapps, we actually see a tendency to support mostly JSON in HTTP based APIs (flanking the original spirit layed down in HTTP1.1 spec).
Maybe because people think its easier, faster, and cheaper to support only one format. This is the case is app rely a lot on manual marshaling, but not so if automated marshaling is put in place.
Cheers, Jorge.
Thanks, Jorge!
Although XML promised a huge number of advantages, reality is that most Web APIs are not using it.
The advantages of JSON over XML make it a popular choice nowadays. I recommend you read this interesting piece comparing both formats: http://www.json.org/xml.html
Pingback: Implementing API Content Negotiation | CodingSc...
Pingback: The Accept Header: A Quick Primer