 |
| Mean temperatures for December Concord, NH, 1871-2007 |
The weekend was beautiful here in New Hampshire but today we’re back to what has over the past five years started to seem like normal for July: cool and cloudy. I’ve been taking an informal survey this month, asking friends whether they’ve gone swimming yet this summer. Almost nobody has, which we all agree is just weird, and which we all tend to attribute to climate change.
For me, our recent pattern of cool and cloudy summers is becoming a deal-breaker. Reliably nice summers used to be the payoff for living here through the winter, but if I can’t count on summer to be reliably nice, I’m tempted to consider other options. That’d be a big decision though, so I’d like to support it with hard data.
Given all that’s been said and written about climate change, it turns out to be surprisingly hard to get hold of historical climate data. I had to look around quite a while before I found this FTP site where NOAA has parked files full of raw temperature and precipitation data.
The files cover the whole world and they go back a long way. Here’s one line from the mean temperature file:
4257260500201993 -37 -73 -2 85 143 190 219 208 164 88 45 -13
I’ve marked the 3-digit country code in red, the 5-digit World Meteorological Organization station number in blue, and the year in green. What follows are twelve values which are monthly mean temperatures in tenths of degrees Celsius.
Since Concord, NH is the closest WMO station to me — 55 miles away — I uploaded the Concord data for mean temperatures (1871-2007) and precipitation (1859-2007) to Many Eyes, and looked for a recent pattern.
I didn’t find one. Here’s a view of summer mean temperatures, and here’s a view of precipitation. Look for yourself.
On the temperature front, we’re all very aware that this past December was freakily warm. And as this view shows, there hasn’t been an above-zero-Celsius December since 1982. But there were more (and warmer) above-zero-Celsius Decembers before the midpoint of the time series — 1939 — than there have been since.
I am not saying that the planet isn’t warming or that the climate isn’t changing. But we ought to be able to explore the evidence for these phenomena, and review the interpretations of them, in much more interactive and collaborative ways. Not only for reasons of global policy, but also so we can contextualize what we see happening around us.
Arguing about the weather has undoubtedly been a favorite pastime of our species since we learned how to talk. Now we can have those arguments in the context of actual data. And as questions about climate change grow more critical, it’s imperative that we do. Hats off to Martin Wattenberg, Fernanda Viégas, and their colleagues for creating Many Eyes and showing how we can.
July 23, 2007 at 1:22 pm
Nice post. I’ve always wanted better access to raw weather data too; this looks promising.
A small point: you might want to update your screenshot of the Many Eyes data to include the legend at bottom, which shows “Mean temperatures (tenths of degree Celsius) in Concord NH, 1871-2007.” After wondering about the unit of measurement in the graph, I at first thought there was no legend published with it until I visited the Many Eyes site itself.
July 23, 2007 at 1:58 pm
Isnt something wrong? Theres no way Concord could have monthly means at -40 celsius. Am I missing something?
July 23, 2007 at 1:59 pm
Aah no, its in 1/10ths of a degree celsius. Stupid me..so -40 is really -4C
July 23, 2007 at 2:17 pm
To see global trending, I’d expect that you’d have to use averages from multiple reporting stations… sample size determines margin of error, and all that.
Even then, it’s more involved than simply averaging the raw data from multiple raw feeds… some typical complications illustrated here: http://www.ncdc.noaa.gov/oa/climate/research/ushcn/
July 23, 2007 at 2:49 pm
It’s been a nice summer here Jon! I’ve been swimming! It’s great excersise and a perfect way to stretch out the body after slouching in a chair in front of a computer and in a car all day. If this summer is indicitive of future summers, than I say fine, except for a bit too much rain. Cool weather makes it easier to work outside.
Anyway, you can’t tell much about *global* climate change by looking at *local* weather change.
July 23, 2007 at 4:16 pm
A line or scatter plot of the mean temperature data is probably more appropriate for spotting trends. Here, it also doesn’t look there there are any strong trends over the time course of the data set, but some particular months do seem to be trending slightly up (August) or down (May) in temperature. Nothing apparent jumps out for June or July, however.
As for whether these past five years have been ‘wierd’ or not, I think it’s just likely to take a decade or two before you can expect to have experienced a representative range of any given location’s climate. Although we talk about ‘average’ weather, it’s not normal for the weather to be ‘average’, at least in the Northeast. What’s normal is for the weather to dance around, above and below the average, from week to week and year to year. I think it’s mainly a failure to recognize this leads people to fret about every excursion away from the climatological mean.
July 23, 2007 at 5:03 pm
I think it was warmer in New England before the recent reforestation efforts… now the air there is maintaining moisture better than before.
July 23, 2007 at 5:56 pm
“A small point: you might want to update your screenshot of the Many Eyes data to include the legend”
Agreed. Done.
“To see global trending, I’d expect that you’d have to use averages from multiple reporting stations”
Agreed. And yet we instinctively react to news about global trends by imputing a connection to local trends. It could, and should. be easier to tease those effects apart.
“If this summer is indicitive of future summers, than I say fine, except for a bit too much rain. Cool weather makes it easier to work outside.”
I should have qualified what I meant by “reliably nice” summer. For me it means a high probability of warm-to-hot sunny days. And since about 2000, it sure feels like that probability is lower than anytime I remember, having lived all my life in either the Northeast or the Midwest.
“I think it’s mainly a failure to recognize this leads people to fret about every excursion away from the climatological mean.”
Agreed. What complicates matters now is that we are pretty sure that there is a long-term trend. Whether or how that signal emerges from the noise of normal (and local) variation is hard, maybe impossible, to know. But again, better visualization of both local and global trends will help us to keep things in context.
“I think it was warmer in New England before the recent reforestation efforts… now the air there is maintaining moisture better than before.”
It’s often argued nowadays that New England is becoming a temperate rain forest. But what’s the basis of that argument? We should be able to see the data, explore how it has been interpreted, and have the option to create alternate interpretations.
July 23, 2007 at 7:59 pm
Jon, while I applaud your enthusiasm as well as the good intentions of Many Eyes, your use of the bar chart (as Tom Pollard pointed out) is not easy on the eyes and is not terribly effective for scientific visualization of temperature and precipitation data (which are data intense and dense). There is also no direct presentation of standard deviations or other useful computed statistical data to make inferences with a human eyeball glance.
Who are the “we” that you refer to, which “we” ought to “review the interpretations” of weather and climate data in “much more interactive and collaborative ways”. “We” is too vague and reminds me of when I hear people use the similarly vague word “they” as in “… you know what *they* say …” Also, could you please describe “much more” and why you believe that there is a dearth of interaction and collaboration such as in atmospheric sciences?
Meteorologists and climatologists have been collaborating and developing tools for shared visualization and interpretation of weather and climate data for over a century, and there has been nothing anti-social or anti-democratic about this process. The Wikipedia / Many Eyes mantra can be taken too far and this so-called need to “democratize” visualization and interpretation by a vague “we” is overblown (at least in the context you have presented herein). Unless, of course the “we” means people who have a shared background. Would you want, for example, a vague “we” to be collaborating over the interpretation of, say, cancer data (visualized or not) without at least some shared background and depth of knowledge (which depth of knowledge has been peer reviewed, tested and perhaps licensed by boards such as by Oncologists)?
Your vague “we” combined with the demonstration of the Many Eyes site trivializes the process of evidence exploration and collaborative interpretation (community of practice? peer review?) with an American 1960s hippy-like grandiose dream of democratization of visualized data that doesn’t need to be democratized in the first place. Did you read the web page at the URI that Bob Drake posted in comments herein? Do you really think that a collective vague “we” is going to take the time to read and understand (or have enough background to understand) the processes presented on that page such as “homogenization algorithms” and what these algorithms mean generally and specifically?
- Brendan Lane Larson
Meteorologist, Weather Informaticist and Member of the American Meteorological Society.
July 23, 2007 at 8:35 pm
“your use of the bar chart (as Tom Pollard pointed out) is not easy on the eyes”
I agree that the line chart’s better for looking at individual months. FWIW, I like using this particular bar chart implementation for clusters of months in combination. Also the histogram to check the outlying years.
“Do you really think that a collective vague “we” is going to take the time to read and understand (or have enough background to understand) the processes presented on that page such as “homogenization algorithms” and what these algorithms mean generally and specifically?”
Your points are very well taken. I have great respect for domain knowledge and peer review. I also have a certain amount of sympathy for the critique of amateurism as articulated recently by the likes of Andrew Keen.
At the same time I really do think that the gap between what science does and what the media says (and what most people understand) about what science does can be significantly narrowed by making the data behind the science, and the interpretation of that data, and the conversations about the interpretations, a lot more accessible.
To turn the question around, do you think we can, as a democratic society, make the kinds of policy decisions we need to make — on a range of issues — without narrowing that gap?
This is not to say that there’s anything anti-social or anti-democratic about current practice in these and other sciences, I don’t believe that. I’m just sensing an opportunity to narrow that gap.
July 23, 2007 at 10:24 pm
[...] interesting stuff from Jon Udell, this time taking some climate data for his area, using the ManyEyes platform and trying to see what has been happening in New [...]
July 23, 2007 at 10:37 pm
in response to Brendan’s comments above, this whole thread is a good example of what can happen when data like this is opened up to more public eyes: Jon’s original presentation and analysis is probably flawed, and a number of people have commented on why. suggestions have been made, criticisms expressed.
ideally the process continues until a decent presentation of the data, and a decent and useful analysis is conducted, and reasonable conclusions are arrived at, taking into account the public exposure of errors etc.
and the reason it’s important is that Jon (or I, or any member of the vague we) may have a pressing issue in need of analysis that the “experts” have not decided to look at or make public (eg, swimming conditions in New Hampshire) … and the only way that we can get to that analysis is to do it ourselves.
which may well help us solve problems that don’t reach the radars of the experts, but are still of great importance to us as individuals, citizens, communities.
July 23, 2007 at 10:49 pm
[...] Eyes, a project at IBM that allows people to analyze and share data in a visual manner. Thanks to Jon Udell’s blog for this tool. It seems to be a below average temperature summer here in Chicago as well [...]
July 23, 2007 at 10:53 pm
Take a look at Swivel (http://www.swivel.com/) … another data visualization tool. Not quite as refined as Many Eyes at first glance ….
July 24, 2007 at 12:31 am
[...] Nobody goes swimming any more « Jon Udell fascinating application of the ManyEyes technology to weather data; need to explore this for the geographies i frequent (tags: manyeyes data visualization jonudell) [...]
July 24, 2007 at 8:15 am
“Jon’s original presentation and analysis is probably flawed”
Not just probably, but almost certainly. And yet…opinions on this topic are routinely expressed in terms of no data at all. So your point is spot on, Hugh, the question is how to begin to have a framework in which opinions and beliefs can be tethered to sources of data. And in which the processes that produce the data, and the interpretations of it, and indeed the expertise that supports valid interpretations, can all begin to be more widely understood and appreciated.
July 24, 2007 at 8:55 am
Jon, I was imprecise in my earlier comment about including the legend. I meant the area below the graph, which includes the unit of measurement (1/10 degree C). I mistakenly called that area the legend. Sorry about that.
July 24, 2007 at 10:23 am
A while back I found a bunch of old Scientific American issues from the mid 50s. They actually go into real depth about the real science (and the articles are written by quite famous scientists… Dyson, Feynman, etc.)
And the covers and illustrations are great, they show real data and real experiments.
Today SciAm is unfortunately a lot more glossy and takes a much shallower/higher level overview on many of the articles (not always). And the illustrations are often just fanciful distractions to fill up page space, they don’t actually communicate any data.
July 24, 2007 at 10:33 am
Oh, hey :) –
from ledgertranscript.com
Take the
Ledger Poll
(unscientific but entertaining…)
Have you been swimming yet this summer?
Yes, many times
[tally] 29%
Yes, once or twice
[tally] 17%
Not yet, but I plan to.
[tally] 38%
No, I don’t swim
[tally] 13%
No, the pool’s been closed
[tally] 4%
July 24, 2007 at 11:31 am
[...] there’s not much subtlety in a recent comment on Jon Udell’s call to experiment with local weather data and look for trends. After reading Jon’s piece on using [...]
July 24, 2007 at 12:37 pm
Although I understand this may be a limitation of the data set you found, I think using average temperature and total precipitation is a mistake. I think you’re really interested in a measure more like number of days with precipitation, regardless of how much actual precipitation fell on those days. Similarly, you really care how many days were warmer than a certain temperature (25C or something like that) rather than the mean temperature.
July 24, 2007 at 2:24 pm
I agree with LeBleu that mean temperature and precipitation don’t describe the weather that well, as people experience it. The daily max, min and avg temperature and daily precipitation are available for Concord at least back to 1921 (from wunderground.com, if nowhere else), but I don’t think ManyEyes offers the tools to do anything useful with daily data, like plot the number of days with temperatures above 80F per year, or number of rainy days, or even average summer (Jun-Jul-Aug) maximum temperature.
July 24, 2007 at 2:31 pm
“I think using average temperature and total precipitation is a mistake.”
Indeed, mean temperature and total precipitation are weak correlates for “days that people (me especially) would feel like jumping in the water.” For that particular purpose you’d want, in addition to what you suggest, hourly readings so you’d know when warmth and absence of participation coincided with daylight hours, weekends, etc. I wonder how far back those readings go, and for which locations?
“Take the Ledger Poll”
Heh. They could at least report the number of responses in the results :-) Still, for late July, that does look like a high percentage of “not yet but plan to” responses.
July 24, 2007 at 7:56 pm
Jon,
A brief follow-up as unfortunately my time is constrained …
> Your points are very well taken. I have great respect for domain knowledge and peer review.
> I also have a certain amount of sympathy for the critique of amateurism as articulated
> recently by the likes of Andrew Keen.
Until you (and fellow Keene, NH resident and fellow Northern Illinois University (NIU) alum Mike Caulfield who posted a comment to this blog post herein) mentioned Andrew Keen’s name, I was not familiar with him and can not intelligently respond to your comment about his critiques of amateurism.
If you look back at my original comment, I never used the words “amateur” or “expert” nor did I try to draw a boundary between what is considered amateur and expert — that’s an entirely separate conversation topic. I merely called attention to the vagueness of the word “we” and also challenged you to quantify what you meant by “much more” in the context of weather and climate informatics re: “much more interactive and collaborative ways”.
> At the same time I really do think that the gap between what science does and what the media says
> (and what most people understand) about what science does can be significantly narrowed by making
> the data behind the science, and the interpretation of that data, and the conversations about the
> interpretations, a lot more accessible.
The data didn’t seem to be a problem (you found it for Concord) and you’ve already paid for it literally (e.g., your American tax dollars siphoned into DOC and its subdivisions — you knew about this right? So the data doesn’t need any further democratization). I’m not sure what you mean precisely by “the media” but in the context of this blog post topic, I presume you mean the broadcasting of weather news (one to many). A one-to-many model doesn’t support “conversations about interpretations” and this is why for years broadcast TV has gotten away with putting just about anyone in front of a TV camera to make weather presentations (I will not touch on whether such presenters are “experts” or “amateurs” as that is a different conversation topic and time consuming, suffice to say that as long as they have the backstop of NOAA for general interpretation there was is very little requirement for skill and interpretation but I surely wouldn’t want to leave global climate change policy up to the unskilled TV weather presenter who doesn’t have enough background in weather and climate). Your question about swimming in the summers in Keene, New Hampshire is too narrow for broadcast news “media”. Sure, the Internet model (many to many) provides the opportunity for motivated individuals to seek out and participate in conversations about any topic but what about skilled interpretation from someone who has background and experience (let alone derivative conversations about such such interpretation)?
> To turn the question around, do you think we can, as a democratic society, make the kinds
> of policy decisions we need to make — on a range of issues — without narrowing that gap?
A “gap” will always exist between people who don’t have enough background (educational and experiential) to make skilled interpretations and those who do. Otherwise, how do the “we” know what questions to raise? I think the commentator LeBleu followed by Tom Pollard exemplified this point when LeBleu suggested that your original question was probably not the best fit question re: “I think using average temperature and total precipitation is a mistake.” It was very nice of LeBleu to raise a further question and therefore educate Jon Udell about a possible error in the question selection process, but what if LeBleu’s question isn’t deep enough (e.g. in this blog post thread and topic there has been no further uptake on “homogenization algorithms” as originally pointed out by Bob Drake, probably because the educational and experiential background of the commentators and the original blog post author ends right there — this is not a personal criticism or an attack on anyone’s egos).
I don’t think its mostly about access (although there are parts of the world and probably parts of America that are yet struggling to become connected to the Interent and Negroponte’s One Laptop program is trying to solve that problem for example). Background and skill (education either academically or experientially or both) requires time and attention which is luxury. I applaud wealthy, philanthropic good intentioned people such as Al Gore who have the luxury of time to create “media” in the form of documentaries, which documentaries were compiled from sources of democratized data and gratis interpretation of said data by a small handful of skilled persons with enough background in climatology (ever try to design and implement a global climate model in Fortran?), with his the hope of raising awareness and reaching motivated people living in democratic societies which people may take action by speaking with politicians to effectuate policy changes that result in a reduction of pollution and improvement of qualit of life on Earth including those who miss the pasttime of swimming during the Northern Hemispheric summers in New Hampshire! :-)
Cheers,
-Brendan Lane Larson
Meteorologist, Weather Informaticist and Member of the American Meteorological Society.
P.S. Note to commentator Caulfield – to have reduced any “xenophobia” you may have had about global climate change, did you not have the option of taking one of David Changnon’s Climatology classes at NIU? http://globe.geog.niu.edu/Changnon_03.htm
July 24, 2007 at 8:39 pm
A belated proof reading correction:
Should read:
… with his hope of raising awareness and reaching motivated people …
and:
… reduction of pollution and improvement of quality of life on Earth …
July 25, 2007 at 9:34 am
“I’m not sure what you mean precisely by “the media” but in the context of this blog post topic, I presume you mean the broadcasting of weather news (one to many).”
No, I mean the process whereby the scientific understanding of climate change is conveyed to the public by way of such mainstream media channels as television, radio, newspapers, magazines.
“Background and skill (education either academically or experientially or both) requires time and attention which is luxury.”
That raises an extremely interesting point. Notwithstanding the American tendency to overwork relative to other advanced industrial democracies, there are a lot of people who aren’t rich yet have lots of time on their hands to watch TV and, more recently, to engage in the production as well as the consumption of various forms of “content” — e.g., movie exegesis on IMDB, YouTube videos, etc.
Time and attention /are/ luxuries, and a lot of folks have the time and the attention for the kinds of luxuries that produce these kinds of shared artifacts.
I like to imagine what might happen if a slightly larger fraction of that time and attention were diverted to subjects like the one we’re discussing here.
Meanwhile, despite all my bitching, it has been perfect summer weather for a few days. Last night after a bike ride I jumped into Spofford Lake and it was glorious!
July 25, 2007 at 1:55 pm
You should try coming to San Francisco and swim at the Dolphin Club. It was 47 degrees during the winter (during the infamous “Polar Bear” swimming season) but is now up to a balmy 65!
July 26, 2007 at 11:05 pm
Jeez. Thanks for giving a pointer to raw climate data. How did you find this ftp site? I’d love to know; I was looking for similar information about a week ago and, based on what I could find on the public website of noaa.gov, thought I had to a) purchase historical data or go to a research library– and I was just looking for daily max/min temperatures in New York City for the past 10 years!
I have to wonder whether the NWS is deliberately making it difficult for the public to access historical weather data.
July 29, 2007 at 6:28 pm
ah, to have no job and the time to go swimming…
August 5, 2007 at 6:48 am
[...] few weeks back Jon Udell blogged about a Many Eyes visualization he had created which charted monthly mean temperatures from 1871 to [...]
August 19, 2008 at 10:05 pm
[...] change – or at least perceived (see some of my inconclusive Many Eyes weather plottings inspired by Jon Udell here) climate change – is a real issue for oyster farmers. While those working the oyster beds used [...]
December 5, 2009 at 10:04 pm
Thanks for taking this. Good message and great you talk about this. I was studying your blog posting.