In the late 20th century, scientists began to flirt with proof that there is dark matter in our universe. As we learned that as much as 96% of the mass of the universe is represented by something we cannot observe, we’re entering a new era of understanding about the nature of matter. However, the struggle in this field is that we only have a measurable knowledge that these things exist, and educated guesses about how much of the universe is comprised of this yet undescribed material.
Integrated software systems using few standards beyond HTTP, better known as Web APIs, now seem to have something in common with dark matter. There are far more organizations building private, or internal-only, APIs as opposed to public API programs. As an industry, this perspective can not be overlooked, and has the potential to change the way we think about developing APIs.

Image credit: http://cosmicweb.uchicago.edu/filaments.html

In the same timespan as the science of dark matter has been vigorously explored, integrated software systems have become so prevalent they are practically commonplace. As the movement into easy-to-understand, HTTP-based APIs (dare I say RESTful APIs) has taken hold, the quantity of integrated systems are exploding.
When we attempt to plumb the depths of this revolution, we come up short on tools for measurement. Sites like programmableweb.com attempt to catalog every public API, by way of crowdsourcing the information. Within the API community, there are various attempts to measure the utility or style of each public API. However, we have no real scientific measure to quantify how widespread API-driven architectures really are.

No charts required

Anyone who follows this space has seen charts plotting the growth of APIs, and the ‘hockey stick’ inflection point we seem to have finally hit. As programmable web.com celebrated the 10,000 API mark in 2013, Andreas Krohn put it best:

“10.000 APIs is huge, but we’ll reach 20.000 in no time, and soon it will be pointless to keep count.”

However, when we put the entirety of the API space into perspective, we’re merely getting a glimpse of the total situation. Programmableweb.com (as well as any other index you can find) only reflects what is visible on the internet. Much like dark matter, we can only make educated guesses as we attempt to find new means of measurement.

“My door is always open”…

Businesses in today’s tech climate rarely expose public APIs. I previously wrote about “5 reasons your API is still private”, describing some of the challenges developers face in convincing business to open their existing private APIs. Competitive risks, additional costs, and the challenges associated with marketing and support are some of the items holding these companies back.

While I didn’t specifically mention ‘education’ as a line item in that post, there is a clearly still a gap. In the past few years, this was perhaps the predominant reason. However, as we seem to have passed into a more mainstream level of understanding, this is probably a far less of an excuse for companies to remain closed with their integration strategies. There are now vast resources available to help educate all levels of business.

The disappearing act

That said, as open API strategies become more mainstream, so do mis-steps which give management a bad taste in their mouth. Any one of the factors listed could force the hands of management to close the doors on their public API plans, perhaps never to be opened again.

At the API Strategy & Practices conference in New York 2013, Daniel Jacobson, Director of Engineering for the Netflix API, disclosed that 99+% of the Netflix API was internal to it’s own use. This was one of many reasons that Netflix decided to stop issuing new developer keys for it’s public API, effectively shutting the door on new external integrations via their REST API.

Jacobson also recently pointed out that keeping your APIs doors closed to the public is often the most appropriate choice in his post titled, “Why you probably don’t need an API strategy”. As the author of the O’Reilly book titled, “APIs A Strategy Guide”, he seems well-qualified to make this statement, if at least from his own perspective of epic scale.

What is measurable?

At the most recent API Strategy & Practices conference in San Franciso, Oren Michels from Mashery, a leading API management provider, helped add insight to the dark mass of the API world. In his talk, he indicated that at least 75% of Mashery’s clients are using API management services for private API integration.
Guillame Balas, CMO of 3Scale, another API management provider, quipped on Twitter about Oren’s disclosure:

@guillamebalas: Amazed that people seem so surprised by ratio of 25% public/75% private APIs. #apistrat

At the same conference, Laura Heritage, an API-focused Product Manager at IBM, indicated similar notions.

@programmableweb: 80% of enterprises understand the need for external API strategy but will start internally and with biz partners @heritagelaura #apistrat

It’s notable that clients who are already willing to invest in this type of platform, probably already have a more significant strategy at play. Clearly there are just as many companies that don’t have a strategy that is this far along yet.

Is the iceberg melting?

As we take in the landscape of the past decade or so of public APIs, we have to wonder what the future holds. Sometimes APIs open their doors to the public and subsequently close up shop. A surprisingly small population have gone to an open model and thrived. Most analysts and experts, including the “API Evangelist”, Kin Lane (serving Presidential Innovation Fellow), agree that in the next few years, open APIs won’t just be a competitive advantage, but a business requirement.
Lane’s perspective comes from assisting in the chore of adding transparency to data of the US government, which is now poised to open up an unprecedented number of APIs for public consumption. It’s hard to deny the coming change toward open data, when even government is making the shift.
We can only surmise how many APIs are already in the works, just waiting to be opened up on the web. This is the “iceberg” that is often alluded to when we survey the current offering of public APIs. However, we should also consider that many APIs will never be fully open to the public, and this closed population may always comprise the majority.

Why do we care?

For those API builders reading who are sure their APIs will never be open, this might feel like salt in the wound. While this post does not provide material steps toward helping open the vast number of closed APIs, there is a takeaway.
As builders of APIs, we should always be focused on sound craftsmanship practices and API design patterns. If all of the current trends take hold, the private API you build today is likely to be the open API of tomorrow. Making quality choices in how you build your currently internal-only APIs may leave you a legacy of quality when the public consumes your work.
In addition to the actual technical construction, business leadership must keep in mind a long-term strategy of the possibilities. Even if the planning is not detailed at this stage, it’s important not to cut corners, as it could ensure your future API too expensive to refactor. I’ve posted on APIUX.com about “Why no one wants to use your API” as a series of warnings to today’s APIs. Most of these warnings are completely valid for internal usage, as well as open APIs.
Keep in mind the potential future of open APIs as you craft your software now, and you may reap the benefits sooner than you think.

Last updated by Jason Harmon at February 6, 2014.