Although the REST community initially took a stance against metadata for REST APIs, a number of metadata standards have none-the-less emerged over the last couple of years, mainly fueled by the need to document APIs for their consumers.
As an added benefit, the same metadata is now often used to generate code (both client and server), create test harnesses, production monitors and perform real-time validation of request and response messages (when applicable). All of these provide a foundation for an improved Quality of Service (QoS) that many enterprises require as they adopt REST for their information architectures.
A short background
Before looking at some of the metadata standards out there, let’s back up a bit and look at what they strive to describe (for those of you that advocate Hypermedia APIs, I’ll try to give some insights on that as well further down).
The common goal of current metadata formats for REST APIs is to specify
- Entry point(s)
- Resource paths
- Methods to access these resources (GET, POST, PUT, etc.…)
- Parameters that need to be supplied with these methods (Query, Template, HTTP Header, etc.)
- Formats of inbound / outbound messages/representations (JSON Schema, XML Schema, Relax NG, etc.)
- Status codes and error/fault messages
- Documentary information (descriptions, etc.) for all these
All these are somewhat derived from the principles underlying REST APIs – i.e. those of resources, representations etc. Non-functional aspects of APIs, like for example authentication (Basic, OAuth, SAML, etc.), security (encryption, signatures, etc.) and versioning are unfortunately still poorly addressed by most REST metadata standards. This is where the WS-* standards “shine”, but it is also what many refer to as WS-(death-star) – indicating the complexity of the WS standards that might ultimately lead to their demise. Hopefully the REST community will align around common standards and best practices more successfully than vendors did in the SOAP-domain around 10 years ago.
Let’s have a quick high-level look at what’s out there (in alphabetical order) – if I missed anything obvious the fault is mine entirely and I apologize in advance. Please don’t hesitate to comment below and put things right 🙂
apiary.io
Apiary has created a markdown-based metadata called the “API Blueprint” format. The parser itself is open-source and the format is well described on their website and on GitHub. Once you have an API Blueprint for your REST API – the apiary platform can be used to generate documentation, create server mocks, perform validation, etc. Although it is technically possible to use the API Blueprint format itself outside the apiary.io platform, most of the value-added functionality in the apiary.io platform seems to be proprietary.
ioDocs
ioDocs is Masherys’ take on REST metadata and documentation; their open-source (node.js based) platform uses JSON-formatted metadata to generate an “interactive documentation” for the described API that can be used to both learn about an API and execute ad-hoc calls against it. As opposed to the other standards described here, ioDocs also includes basic support for signatures and different versions of OAuth. As mentioned, the core ioDocs platform and tools are open-source and free to use outside the Mashery platform, but just like with apiary.io, a large number of value adding features are made available when using the Mashery platform to manage your APIs.
Swagger
Swagger is a REST metadata format being developed by Reverb, a spinoff from the Wordnik team. Swagger uses JSON (although it supports XML) and JSON-Schema to describe REST APis and their parameters and messages. Just like ioDocs it includes an open-sourced UI (swagger-ui, entirely in HTML/javascript) double-serving as documentation and ad-hoc testing utility, but Swaggers’ strength lies in its ecosystem available at GitHub for generating both code for a number of different languages and Swagger definitions themselves via (for example) java annotations.
WADL
WADL (Web Application Description Language) is an XML vocabulary that has been around for quite some time (it least in internet-time). It was submitted to the W3C by Sun in 2009 but hasn’t been standardized on, perhaps due to lack of adoption. Technically it provides good support of the REST concepts outlined above; its grammar mechanism allows for the use of any standard for describing resource representations (JSON Schema, XML Schema, etc.) and there are a number of tools out there to generate code, tests and documentation from WADL specifications.
WSDL 2.0
You might be surprised to see this on the list, but technically both WSDL 1.1 and WSDL 2.0 support description of REST APIs (although the support is meager in WSDL 1.1). Unfortunately though, the lack of “RESTfulness” in WSDL and its association with SOAP will probably result in its inclusion here being shrugged off as a curiosity as opposed to a valid alternative. That being said, WSDL 2.0 does technically support the description of REST APIs but its core model is not around resources/methods/representations, and the tooling is not as extensive as for the above-mentioned formats.
Which one to use?
Selecting the right one for you from the above isn’t easy. Here are some of my personal observations:
- API-Blueprint and ioDocs are spearheaded by vendors that provide a value-adding commercial offering, as opposed to Swagger, WADL and WSDL which are “standalone” formats (for now)
- ioDocs and Swagger provide a standalone open-source tool that you can use to generate great looking browser-based documentation and an ad-hoc testing interface.
- API-Blueprint, ioDocs and Swagger are still somewhat “immature” formats and I expect them to evolve (read “change”) considerably as their adoption and the acceptance of REST principles and QoS standards increases
- Swagger seems to be the one with the most traction in the community
- WADL (which I think is unfairly downplayed) is pretty mature and supports REST concepts very well.
- The REST community should really strive to align around any one of these to avoid the fragmentation and frustration we have seen in the WS community in regard to multiple standards solving the same problem.
When trying to select one of these metadata formats for your API, make sure you take the following into account:
- Is it easily accessible?
- Are the included tools available for your platform?
- Is there relevant tooling available given your target audience and their needs? (Testing, code-generation, etc.)
- Does it support metadata at the level needed in your usage; for example you might decide to create schemas (XML, JSON, whatever) for your resource representations, this is one area where the metadata formats differ.
- Does it support QoS metadata that you might want to make available?
- If it doesn’t support everything you need – is it on their roadmap?
- Is there an associated vendor lock-in for your usage intentions?
This is obviously just scratching the surface, but hopefully gives you an idea of which areas to explore further.
What about Hypermedia APIs?
As indicated in the first sentence of this post, the REST community has been (and still is) somewhat ambivalent to providing out-of-bounds metadata for REST APIs. Wouldn’t it be nice if the API provided metadata about itself and the actions that are available for a requested resource as part of the response/representation itself? Ultimately you would start with just one endpoint which would return links to all related actions / resources allowing you to “drill down” into the API and its exposed resource model – just like a visual hypermedia document can be browsed in a web-browser. This is what is commonly referred to as Hypermedia APIs and HATEOAS (“Hypermedia as the Engine of Application State”) and the debate surrounding its adoption vs. metadata can sometimes take religious proportions – be warned. 🙂
Even though the idea of out-of-bounds metadata is abolished in this construct – metadata in itself isn’t; it’s part of the response returned for a certain resource, and as such it needs to adhere to some kind of “way of doing things”. Let’s have a quick look at two of them.
HAL
HAL (Hypertext Application Language) is a simplistic format available for both JSON and XML APIs to provide linking within a response. A multitude of libraries in many languages are available, both for creating and consuming HAL responses.
JSON Hyper Schema
JSON Hyper Schema provides a mechanism for embedding links in JSON documents, achieving the same goals as HAL in a slightly different fashion. It is part of the JSON Schema initiative, which also includes a large number of tools for creating, parsing and validating JSON documents.
Final words
Ultimately one of the primary reasons for providing metadata is for easing the adoption of an API – be it via generated documentation, code or examples – and as such it has great merit. Reading through the above might have opened your eyes a bit but perhaps not made things easier for you; should you use metadata at all? Which format seems like your safest bet?
Since this post is not meant to provide a specific recommendation, I’ll leave that assessment to you; the only recommendation I will make is that you put your end user first when making your choices – are they invested in some kind of technology? Use that! Do they prefer JSON to XML? JSON it is! Are they legacy and enterprise up to their ears? Perhaps you should use SOAP/WSDL instead of REST (the horror…)!
Most importantly: make them love you and your APIs, with or without metadata. 🙂
Some References (in order of appearance)
- Roy Fieldings original REST principles: http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm
- Apiary.io API Blueprint: http://apiary.io/blueprint
- ioDocs: https://github.com/mashery/iodocs
- Swagger: https://developers.helloreverb.com/swagger/
- WADL: http://www.w3.org/Submission/wadl/
- WSDL 2.0: http://www.w3.org/TR/wsdl20/
- HAL: http://stateless.co/hal_specification.html
- JSON-Hyper-Schema: http://json-schema.org/
This is one of those perspectives that reminds us we are just at the edge of a new frontier. The liberty of choice is presenting us with so many options that the best is not yet clear. While standards helped drive the SOA adoption in the SOAP era, our lack of standards is driving more innovation. I like that you touched on meta as a topic, but I’m sure some folks will take issue with referring to Hypermedia links as meta (versus being a direct aspect of the content).
Great stuff!
Pingback: An Overview of REST Metadata Formats | Next Web...
Hi Jason,
Thanks for this! Regarding if links/actions are metadata or not – perhaps there is an academic distinction as you say, but personally I find that describing and providing available actions for a resource inline (as part of the response) or externally (for example in a swagger definition) is still the same “metadata” and there are pros and cons for each approach (maybe that would be a good follow-up post!?) – once again knowing your end users and how they will want to consume your API is crucial to making the right choice.
cheers!
/Ole
Pingback: Scott Banwart's Blog › Distributed Weekly 202
Another way to view this space is to think about the differences between
– a Service document approach (WSDL/WADL, etc),
– a Discovery document approach (JSON-LD/Hydra, Nottingham’s JSON Home documents, Google’s Python library, etc.)
– a Hypermedia approach (Collection JSON, HAL, Siren, etc.).
While they all attempt to expose metadata about a service or API, they not only do this in different ways, but rely on different optimizations. Service docs optimize the build experience (via proxy generation). Discovery documents optimize the runtime via configurability; essentially Discovery documents are consumed by a static runtime client. Both of these approaches favor “knowing” the entire set of possibilities are design/build-time.
Hypermedia is a runtime optimization based on _not_ knowing the entire set of possibilities. Hypermedia metadata is served up on an “as needed” basis – a very diff approach and one that continues to vex those attempting to create documentation for APIs.
Nice article. It deserves an update, as the scene evolved a little. Should add RAML to the analysis.
Thanks – I agree; both RAML and API Blueprint deserve mention – and there are quite a few Hypermedia related metadata initiatives as well…
/Ole
I’d also say that some options are better than others for a design first approach.
Pingback: Data Modeling for APIs. Part 2: REST and JSON | Linked Data Orchestration
Where do we stand in 2015?