Where the oil comes from: Not from where I thought

At a party the other night, a friend mentioned that the country supplying us with the most oil is Canada. Maybe so, I said, but on a regional basis the Middle East dominates, right? He wasn’t sure, but didn’t think so. And it turns out he was right, at least according to the US Dept. of Energy data he sent me. That data says that the Middle East ranks third among our regional sources, behind North America and Africa.

Here’s the world overview for 2007 in thousands of barrels:

And here’s the regional breakdown:

North America 1,648,765 33.56%
Africa 980,231 19.95%
Middle East 837,841 17.05%
South America 784,999 15.98%
Europe 567,152 11.54%
Asia 91,236 1.86%
Oceania 2,774 0.06%

The links go to regional views where you can hover to reveal per-country numbers.

When I do these kinds of exercises, I’m always struck by two things. First, it amazes me how much of what we think we know is wrong. I was sure that the Middle East was the dominant regional source.

Second, I’m always a bit discouraged by how geeky you still have to be — even with the great online tools we have now — in order to pull answers to simple questions out of raw data. When my friend cited these numbers, the first thing I wanted to know was: How do they break down by region?

I wound up using Dabble DB because I happened to know that it includes all the necessary ingredients:

  • Can import tabular data from web pages
  • Can drop and rename columns in an imported table
  • Given a column with locations — countries, states, zipcodes — can map the corresponding columns
  • Can publish views for anybody to see

This was a huge leg up! But a lot of folks wouldn’t know about that tool. And even if they did, many wouldn’t overcome some of the remaining obstacles. For example:

  • Importing. There are a few different ways to grab data from a web page. You can have Dabble DB parse the page, or you can copy/paste. In this case, I wound up trying both and had better luck with the latter. But we’re still very much in an era when data published to the web is not really intended to be used as data. That first step can be a doozy.
  • Sharing. After pasting in the data and reducing the table to two columns — country names and 2007 1000s of barrels — I had my answer. And if you were an authorized user of the application, I could have shared it with you. But in order to publish to the world, I had to produce a special URL. And then I realized a single one wouldn’t suffice. The shareable views aren’t interactive. You can’t drill down from the world overview to the Middle East segment. So I wound up having to create views for each region, generate an URL for each view, and keeping track of all that was confusing even for me.

Still, I’m excited. We’re really close to the point where non-specialists will be able to find data online, ask questions of it, produce answers that bear on public policy issues, and share those answers online for review and discussion. A few more turns of the crank, and we’ll be there. And not a moment too soon.

57 thoughts on “Where the oil comes from: Not from where I thought

  1. Pingback: The Middle East supplies most of the USA … « Paul M. Watson

  2. Tim G

    I’m sure you’ll get this from about 30 people, but:

    Oil is fungible. It is the same no matter where it comes from. The reason we get so much oil from Canada is because it is so darn close. The Middle East is close to Europe and Asia. So it makes sense for their oil to go to closer destinations.

    It would not be cost-effective for Canada to ship oil to Europe while the Middle East was shipping it to us.

    –tim

    Reply
    1. kala

      actually oil is not the same no matter where it comes from. the best place to get oil is from africa because it is the richest. it doesn’t have to be as refined like it does from other places. the less it is refined the better it is and the more our car and other things that need oil like it. also the more refined that it as to be the more it cost because more work has to go into the oil

      Reply
  3. Adam Fisk

    This topic has fascinated me since Exxon Mobile oddly included similar data in a New York Times ad a couple of years back.

    @Tim If you were an alien visiting earth, this map would make sense for the reasons you say. The point is that common perception differs so markedly from reality — a dangerous disconnect when that perception is so pivotal in determining public policy.

    The data visualization topic is equally as fascinating, and DabbleDB looks fantastic. I know many organizations that are forced to compromise the quality of their work because they don’t have the technical skills to pull off what DabbleDB makes so easy. Exciting stuff. I could see Google snatching up a company like that very quickly.

    Reply
  4. Jon Udell Post author

    “The point is that common perception differs so markedly from reality — a dangerous disconnect when that perception is so pivotal in determining public policy.”

    Exactly. In kitchens and living rooms everywhere, we talk about these kinds of issues every day. But now we’re all getting access to the kinds of data, and the kinds of tools for making sense of the data, that were formerly available only to the experts and specialists we thought we had to trust with decisions.

    Now if we can just make those last-mile connections. In this case, it probably took me 45 minutes to use DabbleDB and WordPress to create and publish that regional breakdown. But I can absolutely envision the day, soon I hope, when it would only take 5 minutes, would be quick enough to do the fly during the actual discussion, and would be easy enough so that you wouldn’t have to have somebody like me in the room to do it.

    Reply
  5. SDC

    ManyEyes by IBM is a nice somewhat social data visualization site. It doesn’t adress your need to pull tabular data in from websites, though.

    I find Freebase and their MQL (Metaweb Query Language) interesting, however it depends heavily on their data being complete and correct, and MQL is not for the faint of heart.

    (sigh) Someday…

    Reply
  6. Ali

    “The point is that common perception differs so markedly from reality”

    I must be missing the point, but .. I thought Tim’s point was that it doesn’t matter where the oil to a particular nation comes from. The common (mis-)perception doesn’t alter what makes the ME a “region of interest”. If Saudi Arabia decides to restrict supply, the price of oil everywhere goes up, regardless of how much we get from Canada. No?

    Not that I’m advocating continuing false perceptions. The “right” answer for the wrong reasons is still wrong, IMHO. I guess I’m just curious how you think this particular information might affect policy in the large.

    Reply
  7. Chris

    To tell an even deeper story I think a good follow-up to the “region of origin” chart would be one that focuses on proved oil reserves by region:

    http://www.eia.doe.gov/emeu/international/reserves.xls

    According to these estimates, the Middle East region has over 700 billion barrels (over half of the entire global reserves) while the North American region has somewhere between 58-211 billion barrels. The huge gap in the N. American estimate comes from Canada’s significant “oil sands” potential that oil surveys did or did not include based on individual survey criteria.

    So, yes, while most oil imported to the U.S. currently comes from Canada, I think this statistic will only be valuable for short-term purposes and shouldn’t change the perception of the Middle East as a prime time player.

    As more tools like Dabble DB become available to help us visualize data beyond rows and columns (thank goodness) I think it becomes especially important for all of us who view data to use a critical eye when trying to decipher meaning and any broader implications.

    Reply
  8. John Loughlin

    “…common perception differs so markedly from reality”

    One reason for this is not because of a lack of data, or the tools that allow us to group this information easily, it is because authors have agendas, and we’re lazy.

    When you visited the Energy Information Administration website

    http://tonto.eia.doe.gov/energy_in_brief/foreign_oil_dependence.cfm

    You would have read:
    “Some may be surprised to learn that almost 50% of U.S. crude oil and petroleum products imports came from the Western Hemisphere … We imported only 16% of our crude oil and petroleum products from the Persian Gulf…”

    and going forward:
    “Our dependence on foreign petroleum is expected to decline in the next two decades.”
    and
    “The Energy Information Administration (EIA) projects U.S. crude oil and petroleum products imports will hold approximately steady in the next two decades.”

    However if you went to Google and searched for:
    foreign oil dependence site:.gov

    “removing barriers to domestic production of oil and gasoline”

    http://www.whitehouse.gov/news/releases/2008/06/20080618-4.html

    “Most of the world’s oil reserves are concentrated in the Middle East”

    http://www.fueleconomy.gov/FEG/oildep.shtml

    “growing dependence on Middle East oil” and “dependence on the Middle East is projected to increase by leaps and bounds”

    http://tinyurl.com/5hdprs

    You may also have read:
    “My friends, I will have an energy policy that we will be talking about, which will eliminate our dependence on oil from the Middle East that will prevent us from having ever to send our young men and women into conflict again in the Middle East,”

    http://elections.foxnews.com/2008/05/02/mccain-says-100-years-in-iraq-ad-is-direct-falsification/

    and

    “Problem is, we get oil from some parts of the world and they simply don’t like us,”

    http://www.foxnews.com/story/0,2933,212063,00.html

    Is it any wonder that you didn’t know that 33.56% of our imports come from Mexico and Canada (almost twice the amount from the Middle East)?

    I fully support and encourage the use of Dabble and other tools to massage this information to make it more accessible, and together with Google and YouTube easier to find and more fun to devour, but let’s also remember that we need to still encourage intellectual curiosity, critical thinking, and hard work.

    Reply
  9. mike simonsen

    I’m always a bit discouraged by how geeky you still have to be — even with the great online tools we have now — in order to pull answers to simple questions out of raw data.

    This isn’t really a tools problem. It’s an expertise problem. And it’s ok. It’s why “information wants to be free” but analysis is revenue.

    Reply
  10. Matt Thompson

    Your data analysis is misleading. The political focus is on crude oil, not “all” petroleum products (which is what you’ve mapped). Your data source (US Dept. Of Energy data) shows “crude oil” import sources as well (not mixing the pet. products).

    From this you can see it’s actually not just Canada, it’s Mexico as well (virtually the same historical reliance on both for crude oil, w/the exception of last year when Canada starting pulling ahead). Saudi crude is of similar magnitude (hence the interest in the ME region).

    Petroleum products are cheaper when engineered near the crude source, but can be efficiently created anywhere along the supply line (even here in the US, where much of it is done).

    Do your homework before you write…

    Reply
  11. Bertil

    As pointed out, Middle East mostly exports to Europe; they also have much larger stocks, and can increase capacity more or less at will; in spite of the turmoil in Iraq, Saudi Arabia is far more stable then Eastern Africa countries, when “protection” is a tricky business — and Europe and China are having murkier stands.

    Reply
  12. Pingback: tecosystems » links for 2008-11-10

  13. Pingback: Sean’s Mental Walkabout » Blog Archive » Links for November 10th

  14. William Cox

    Nice map, Jon.

    A non-specialist with rudimentary research skills, using only the Internet, could find this information today. The tools to transform that data are, as you note, becoming easier to use.

    But they first have to ask the question, and be curious enough not to take the first answer at face value.

    There’s a lot of free data on the Internet, if only you ask the question that makes you look for it.

    Here’s a follow-up question: who produces the most oil?

    Reply
  15. Stephen G

    This photo shows country size proportional to its reserves.

    Looking forward to the day when what the map tells us becomes inconsequential.

    Reply
  16. Pingback: Where the oil comes from: Not from where I thought « CriticalMusings

  17. Patrick Foley

    My understanding is that the point of PopFly is to take these parts and pieces (like DabbleDB results) and put them together in meaningful ways so non-technical (or at least semi-technincal) people can share their work. Do you think it has a place in problems like this?

    Reply
  18. Jon Udell Post author

    > Do you think it has a place in problems
    > like this?

    Absolutely! I would like to be able to use it here, but wordpress.com won’t permit iframes.

    Reply
  19. Pingback: The Week Links in the Chain I « The Committee of Public Safety

  20. Pingback: Cloudy Thinking » Blog Archive » Where does majority of U.S. oil come from? CANADA.

  21. Pingback: Visual numeracy for collective survival « Jon Udell

  22. Steven Vore

    “The Middle East” and “North America” are hardly the same when it comes to area. It seems somewhat arbitrary to base our understanding of natural resources on lines drawn by political entities or cartographers. I wonder what a map would look like with all the earth’s surface broken into same-sized (perhaps “Middle East”-sized) chunks.

    Reply
  23. Pingback: US Oil Doesn’t Come From Where You Think it Does | FlowingData

  24. Pingback: US Oil Consumption by Region | The Big Picture

  25. debreuil

    I mentioned this when John McCain said “we send 600 billion dollars a year overseas, to countries that don’t like us very much”. For the record, Canada isn’t overseas : ).

    (joke!)

    Reply
  26. Matt

    @Steven Vore

    Those arbitrary lines are very important when discussing natural resources, when the denote extreme differences in political and cultural orientation, legal regime, and access to said resources.

    Reply
  27. Pingback: Unrequired Reading {20.11.08 to 21.11.08} | Adrian Monck

  28. Christian Nally

    This is fantastic use of a tool to simplify and present data.

    The points made about Middle East oil going to Europe are good ones. Does the EU have the equivalent of the DOE? (i.e. can we get that data too?)

    Even better, could this graph be made to drill down? That is, when we click on a region (say Africa) we then get the list of providers within Africa?

    Great job!

    Reply
  29. Mike

    “and would be easy enough so that you wouldn’t have to have somebody like me in the room to do it.”

    Dont wish yourself out of a job. I am not sure why anyone in their right mind would want that simplicity to go out to everyone. People who depend on doing such work for a living will now have their boss’ 10 year old son to compete with. Not good. Its bad enough that corporations are throwing great programmers to the unemployment line and hiring people in India and China for 1/4 of the pay.

    Then the local, state and federal governments wonder where the jobs are and how to create more jobs and, best of all, where has all the tax revenue gone? Oh, they’ll just raise our taxes.

    Its not good. The world needs ditch diggers too.

    Reply
  30. Pingback: The Enlightened Despot » Blog Archive » My worldview is now destroyed

  31. Jon Udell Post author

    “The data for the U.S. is not correct: you used the imports from the U.S. Virgin Islands (119,607 thousand bbl/year in 2006), but the map makes it seem like this is U.S. production.”

    Heh. That’s a great point, thanks. I omitted domestic production because the idea was to show imports. So U.S. should have been zero, but Virgin Islands counts as U.S. so its contribution shows up.

    The 2007 number for the U.S. was close to the 2006 number you cite: 1.8 million in thousands of bbl.

    Perhaps the title should be Sources by Region, and the data should include U.S. production? In that case, the breakdown would be more like this:

    North America 3506109 51.79%
    Africa 980231 14.48%
    Middle East 837841 12.38%
    South America 784999 11.59%
    Europe 567152 8.38%
    Asia 91236 1.35%
    Oceania 2774 0.04%

    Reply
  32. Jon Udell Post author

    “People who depend on doing such work for a living will now have their boss’ 10 year old son to compete with.”

    There is no upper bound on potential breadth, depth, and quality of work that can be accomplished in this realm, or in any realm.

    When we make it easier to do the basic stuff, the skills formerly needed to wrestle with now-unnecessary complexity can be valuably deployed wrestling with still-necessary complexity.

    Reply
  33. Pingback: Where Oil Comes From | Green Taxi

  34. Pingback: Cloudy Thinking » Blog Archive » Jon Udell’s visualization of oil sources. Surprising.

  35. Pingback: data mining for oil sources « Sarah Allen’s Weblog

  36. W^L+

    What your data misses is that a good portion of the imports from Africa are comes from Arab nations in the Sahara. Not Middle East, but definitely allied with the ME.

    Reply
  37. checkle

    The reason people connect mass oil importation and the Middle East, I think, is because there are few countries in the area, it is a relatively small region, and it is very politically heated. Also, Saudi Arabia is the 2nd most contributing country to U.S. oil behind Canada. I would like more stats but this conclusion comes from the EIA website. What I want to know is the yearly oil importation from Saudi Arabia alone so that I can compare it to each region. This may help explain the connection if it measures up to entire region importation.

    Reply
  38. Izuako Tobechukwu

    pls somebody should send me materials on this information. it is for my research work for Masters Degree programme.

    1. Tabular statistics of United states oil supplies from 1990 – 2006

    2. Statistical data of U.S military occupation and role in the middle East 1990 -2006

    my mail is skolo2door@yahoo.com

    Reply
  39. Pingback: Time, space, and data « Jon Udell

  40. Pingback: A conversation with Andrew Turner about data and design in the geospatial realm « Jon Udell

  41. Pingback: GM and Government Ownership Question - Page 5 - Pontiac Solstice Forum

  42. Schipholtaxi

    That governments and oil companies manipulating oil prices is not new anymore. Self-wide wars caused by oil. It is time that more use of other raw materials used. In Amsterdam next year to the first taxi ride on the rapids.

    Reply
  43. Taxi

    Amsterdam, Holland is a precursor to that kind of thing. Taxis are a major problem because they produce many bad exhaust. Stron would be the solution!

    Reply
  44. Pingback: We don't rely as heavily on the Middle East as I thought we did - Conservative Republican Discussion Forums

  45. Lee De Cola

    The table seems sufficient. The numbers could be divided by another 1000 and a total (which doesn’t agree with the map legend) would help. The % column could be to 1 decimal place and also needs 100%.

    Reply
  46. Pingback: Canada Used Furnture

  47. Pingback: IT Support for small Businesses in Maidenhead

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s