Should I visualize more weather data, or just build an ark?

Last July the crazy weather prompted me to examine my subjective notion that things were, indeed, going crazy. However, a cursory analysis of NOAA’s historical temperature and precipitation data, available back to 1921 for the Concord, NH station that’s closest to me, didn’t seem to reveal any notable patterns.

Well, we’re having another crazy summer, and I took another look at that data. This time around I used Excel to do some deeper massaging, and here’s what I’ve found:

Monthly precipitation for Concord NH, 1921 to 2008

(top 15%)

rank jan feb mar apr may jun jul  aug (1-10)
1 1979 2008 1937 2007 2006 1944 1938 2003
2 1978 1981 1936 1936 1984 2006 1940 1973
3 1935 1951 1953 2004 1954 1998 1988 1953
4 1958 1984 1983 1933 1940 1922 2008 2008
5 1999 1970 1942 1996 1945 1982 1928 1946
6 1936 1971 1925 1983 1922 1935 1967 1980
7 1953 1955 2008 2005 1968 1945 1951 1937
8 1996 1929 1933 1951 1937 1968 1981 1977
9 1986 1969 1994 1940 1983 1987 2007 1969
10 1952 2003 1922 2000 2004 2001 1972 1943

I ftp’d the data from ncdc.noaa.gov/pub/data/ghcn/daily/all/USC00271683.dly, and used Python to parse the fixed-length records into a tab-delimited format like so:

year month day precipitation

Then I brought it into Excel. I’m no kind of Excel wizard, so there are surely better ways to proceed, but here’s what I did. I started with a pivot table where values are the amounts of precipitation, row labels are years, and report fields are month and day. Next I reverse-sorted the rows by amount of precipitation, and restricted them to the top 10 years. Then I filtered month by month — and in the case of August, by just the first 10 days for which 2008 data is available — and copied each month’s top 10 years of precipitation into the final table. Finally I selected that table and used conditional formatting rules to colorize the top 15% of year/month combinations.

Wow. It really has been a snowy and rainy decade around here. And 2008 takes the cake.

Are there easier ways to do this with NOAA’s climate data? From time to time I look, and I never find anything. Maybe I should turn this spreadsheet into an application that’ll search NOAA for your local station, grab your temperature and precipitation history, and visualize it like this.

Or maybe I should just go and build an ark.

3 Comments

  1. Thanks Jon for the climate data I need all of the data I can get hold of. I’ve munched the ECAD precipitation data but the more the merrier. For my tests I’ve used the FileHelpers .NET library to process 19 million precipitation records from the whole of Europe. I’m doing an open source time series database in c# and need as much data as I can for testing purposes.

    Update: can’t get the data from NOAA using the link you provided :(

  2. As an aside, are you permitted to build sites using this data, from their website I can only find limited samples and the expectation that you’re gonna have to pay for anything more.

    Seems odd that as the public purse pays for the climate data to be collected that the information isn’t available free. I’ve had my own run in with the UK meteorological service about this very thing…UK data isn’t freely available either.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s