The World Bank’s web of data could be webbier

As Stefan Tilkov notes tearfully, the REST API for the World Bank data leaves something to be desired. The URI to fetch the list of countries looks like this:

http://open.worldbank.org/rest.php?method=wb.countries.get

In my review of the Leonard Richardson and Sam Ruby book RESTful Web Services I summarized their best practices for making a service “look like the web”:

  • Data are organized as sets of resources
  • Resources are addressable
  • An application presents a broad surface area of addressable resources
  • Representations of resources are densely interconnected

So in this case, the resource that is the list of countries might simply be modeled as:

http://open.worldbank.org/countries

Likewise, given that the indentifier for electric power consumption is EG.USE.ELEC.KH.PC, and for China is CHN, you might have:

http://open.worldbank.org/indicators/EG.USE.ELEC.KH.PC/CHN

Or, for all countries:

http://open.worldbank.org/indicators/EG.USE.ELEC.KH.PC

Or, to fetch all indicators for China:

http://open.worldbank.org/countries/CHN

If the data were organized this way, and if results included URIs for traversing up and down, you could explore this web of data by just navigating around in it.

When you draw attention to this question of style, you risk seeming like a pedantic REST philosopher. But it’s really just a practical, down-to-earth kind of thing. If I’m a developer exploring a collection of data, it’s convenient to be able to navigate a web of linked resources, and have the API revealed to me as I go along.

18 Comments

  1. This is nothing to do with being pedantic. This is to do with following sensible principles laid down by sensible people. I’m stunned that people are still declaring services as RESTful when blatantly they are not.
    Acually no…I’m not stunned at all. I’m resigned to stupidity like this!

  2. Agree with you & Jamie. It isn’t pedantry at all. It’s a shame because with a pretty simple apache re-write I’m sure that the data could be presented in a much more discoverable & understandable way. They’ve done the hard work, all that was required was a bit of finesse on the presentation.

  3. > It isn’t pedantry at all.

    Agreed. I’m just saying that criticism should be accompanied by a reminder of why the preferred way is preferred.

  4. Hello Jon,

    My friend Nat was wondering about this data the other day, and since this is all static data the question is: why was this data not published merely as a Sqlite database dump?

    Do we really need a REST web service to static data when we already have a good query and mining language?

    Maybe the only use of this REST API is to download all the data and stick it into a SQL database ;-)

  5. Once nice aspect of good URI design and a REST approach that deserves the name would be that you couldn’t tell whether the data is kept statically, generated dynamically, or some arbitrary mixture.

  6. > Do we really need a REST web service to
    > static data when we already have a good
    > query and mining language?

    I had the same thought, particularly given that there’s not a huge volume of data here. An all-in-one dump would suit me just fine, as an adjunct to the API calls.

    If the data were volatile you might rather let the API serve the freshest, but this doesn’t seem to be volatile stuff.

    It does seem that creating an API often boils down to cloning what somebody else did, without regard for the specifics of the case.

    But, this shouldn’t come across as: “So we released our data, and all RESTafarians did was bitch and moan about the URI syntax.”

    It needs to be clear that yes, we are grateful to have this access, and we can and will work with it, but just FYI there are some convenience issues in the packaging you might want to look into.

  7. > Maybe the only use of this REST API is to
    > download all the data and stick it into a
    > SQL database ;-)

    Although, it might be a good candidate for an Astoria-like (ADO.NET Data Services) treatment, with navigable/queryable projections of indicators and countries.

  8. It’s not neccesarily about REST, it’s about the usual lazy WWW practice of exposing your implementation details (“rest.php”) in the interface. Now everyone is screwed when they reimplement it using something other than a PHP script that happens to live in a file on the webserver called “rest.php”.

  9. Hi folks,

    I am the one behind this API, not as a developer but as a “business” guy (i hate the term). I have since moved on to a new job but, in defense of the Bank wanted to let you know that this was done quickly, with little resources and that it is very much a work in progress. The bulk of the work went into aggregating the various DB, coming up with a sound database structure etc…the feedback in this thread on how to make it more useful is definitely helpful and am sure my former colleagues at the bank will take it into account.

    PG

  10. Hi Jon,

    More important than the specifics of the Bank, is the concept of an API enabling you to navigate a web of data. It’s a key point I hadn’t seen before. Thanks for the insight!

    KC

  11. Does anyone know of a data dump of these static data available in SQL or some other format? I don’t have the skills to develop a web ap, but I can play with SQL or other db applications to give me ‘access’ – pun intended.

  12. Hi everybody, please take note that we released a greatly improved version 2.0 of the API back in April 2009, more details here: http://www.agileapproach.com/blog-entry/world-bank-api-20-launch. We continue to make improvements to the API including adding more data, languages, and query types. For more, check out http://data.worldbank.org/developers – here you’ll find that we actually now have 2 APIs, one for indicator data and another for project data. You will also find a graphical API query builder to play with. If you have any comments or suggestions please don’t hesitate to contact us directly at data AT worldbank.org

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s