Back in 2006, Jeff Lindsay proposed a different way of consuming Web resources that would eliminate the need for constantly polling APIs for changes. This new pattern was called webhooks and has since been adopted by companies such as GitHub and Google.

The main advantage of the webhooks pattern is that your application doesn’t have to make periodic calls to APIs while it’s waiting for changes. Instead, APIs will call your application on a specific endpoint informing that something interesting has happened. What’s missing is a way to programmatically tell APIs that you’re interested in receiving calls and registering endpoints.

Alternatives to webhooks

As the webhooks pattern gained popularity and more companies started using it, it became the de facto standard way of implementing asynchronous API requests. So much that, in 2009, it was used as the basis of an open publish/subscribe protocol called PubSubHubbub.

PubSubHubbub, or PuSH, has the goal of letting parties “get near-instant notifications (via webhook callbacks) when a topic (resource URL) they’re interested in is updated.” The protocol has gone through several revisions and the latest one, 0.4, is from June 2013 and is backed by numerous individuals and companies.

The  PuSH protocol’s main features let a caller subscribe to and unsubscribe from topics. It also has a number of mechanisms to control subscription validation, verification of intent and authentication of content distribution.

Another approach to solve the subscription problem is the HTTP Subscription Specification, whose main goal is to extend “HTTP to support subscribing to events based on HTTP callbacks (webhooks).” The specification is backed by Jeff Lindsay and folks from GitHub and its latest revision is from February 2013.

The main difference between PuSH and the HTTP Subscription Specification is that PuSH offers validation, verification of intent and authentication on top of the basic subscribe and unsubscribe operations.

To summarize, PuSH offers a simple way to programmatically subscribe and unsubscribe to notifications of events happening at a server while taking care of security issues like validation, verification of intent and authentication. In short, PuSH is an open protocol that treats webhooks like subscriptions.

REST well

While It makes sense to follow the Representational state transfer approach when dealing with operations that directly map to application objects, in the case of subscription management there is no direct advantage in doing it.

PuSH handles the complexity of managing subscriptions in a very simple way, defining the following subscriber operations:

• Subscription request (for creating a subscription): HTTP POST to the subscription server URL passing the callback, the topic and hub.mode parameter set to “subscribe”;
• Unsubscription request (for deleting a subscription): similar to the subscription request but setting the hub.mode parameter to “unsubscribe”.

In addition to these operations, the subscription server may verify the intent of the subscriber by making an HTTP call to the provided callback with the same parameters requested before and an additional hub.challenge that must be echoed back by the subscriber.

Fat is better

After a subscription has been created and validated the server will start pushing information about interesting events down to the subscriber. These calls, usually called pings, should give the subscriber enough information about the event.

These pings can contain less or more information, depending on the server implementation. If a ping payload doesn’t contain enough information, the subscriber needs to make an additional call to the server, asking for all the information about the event.

A skinny payload has 3 disadvantages, from the server point of view:

1. Latency: a skinny payload requires at least 3 hops: 1) server informs subscriber of new event; 2) subscriber asks server for event information; and 3) server returns event information.
2. Bandwidth: a skinny payload consumes more bandwidth as the server has to address two additional calls (see Latency above).
3. CPU usage: a skinny payload consumes more CPU as the server has to handle the call from the subscriber asking for more information, with additional cost.

From the subscriber point of view, a fat payload is always preferred, unless a high percentage of pings are misses, in which case it’s better to stop using a publish/subscribe model altogether.

The Web is Open

Initiatives like webhooks, PubSubHubbub and the HTTP Subscription Specification contribute to a more Open and interoperable Web and should be embraced and adopted whenever possible.

If you’re a company working with different APIs you should definitely follow the standards and contribute to making them better.

Please join in the discussion on this article over on Hacker News and reddit.

Last updated by Bruno Pedro at September 12, 2013.