Actually not, and I appreciate your thoughtful comments very much. Although the long-term trend is clearly toward more transparency, there are some outcomes we’ll reach more easily than others, and some we shouldn’t even aspire to.

In one sense there’s nothing new here, it’s all been thought about before. But changing circumstances now warrant rethinking. I’ve written a lot in the past about what happens when the notion of public information changes from “available to those with time and money to visit city hall” to “available to everybody”.

Similarly, there’s about to be a sea change in terms of what constructive /or/ destructive uses can be made of public information by empowered individuals wielding modern communication and information technologies.

As a society, we need to be having a conversation about what are the new risks, what are the new opportunities, and how should we proceed.

‘Of course, in the London cholera outbreak the presumptive expert was William Farr who had reached the wrong conclusion. And yet he released data that enabled John Snow to get the right answer.’

Yes, but sickness and illness are, largely, acts of nature (* – see below). Crime is committed by individuals, and the perception of crime is a dangerous thing, so can lead to some pretty ugly incidents

http://www.guardian.co.uk/child/story/0,7369,361031,00.html

Publishing any form of crime stats can be very dangerous, hence court restrictions on reporting and the like. Setting the police agenda by ‘public perceptions alone’ tends to lead to poor policing due to the short-term nature of (often media led) frenzies and scapegoating.

I think I’ve probably led you a distance from your original point, let me say that I take your larger point about getting data and analysis into the hands of more people (sort of ‘open source’ for data analysis), it’s just that crime is a particularly dangerous statistic to play with, see also publishing census data, immigration statistics, HIV rates (c.f. ‘sickness and illness’ above) and anything else that allows an aspect of moral outrage of one set of values over another. Pogroms and the like are an ugly and predictable aspect of human nature.

–
Tim

http://www.mattmcalister.com/blog/2007/05/17/175/crime-data-stories/

And of course googling ‘crime maps’ yields lots of examples.

Excellent point. This can lead to discussions of whether, or to what degree, to aggregate data spatially, temporally, or both.

There’s also the perenially interesting question of ease of access to information that /is/ public, but not aggregated. A patient compiler of the crime column in the newspaper could over time amass the same data as might more conveniently be published in aggregated form. Does not publishing amount to a “security by obscurity” argument? If so does that argument hold some water? Tricky questions indeed.

“the proper interpretation of some of the data is best left to the experts who might know how best to avoid simplistic interpretations…”

Another excellent point. Of course, in the London cholera outbreak the presumptive expert was William Farr who had reached the wrong conclusion. And yet he released data that enabled John Snow to get the right answer. Expertise can and does exist in many places.

http://www.i2.co.uk/Products/Analysts_Workstation/default.asp

as part of their analysis of larger patterns (it’s part of a suite of tools more typically aimed at the analysis of individual cases).

The buzz-term for this is Crime Pattern Analysis, but crime is typically much more dynamic than a simple GIS approach might show – for example, you have to look at the influence of time on clustering (mugging sprees follow people, people’s movement patterns change according to the time of day, day of the week etc.). Much of the interesting information is not suitable for publishing to the public (it’s not just crime stats but also lots of intelligence gathering in the process of ongoing investigations). And the dangers of publishing subsets of data include issues such as tipping off criminals as to what’s detected and what’s not (and how timely the reporting is), tipping off criminals as to what crimes may be financially viable, and identifying individuals as victims (“this house was robbed 6 months ago, I bet they’ve got a load of new stuff by now”).

The Washington police have a particularly pro-active and dynamic approach to this sort of analysis. But this involves answering to authorities for prioritisation of tasks – the public perception and concerns regarding crime is one thing, but the proper interpretation of some of the data is best left to the experts who might know how best to avoid simplistic interpretations of complex, incomplete, multi-variate data (Law Enforcement Analysis is a specialist role, this isn’t something routinely undertaken by beat cops and the like).

Regarding doing something useful with the data, good luck, though others have had some success.
http://www.google.com.au/search?client=firefox-a&rls=org.mozilla%3Aen-GB%3Aofficial&channel=s&hl=en&q=google+maps+crime&meta=&btnG=Google+Search

Certainly the advent of mapping APIs is part of the enabling process.

Problems may arise on the borders of jurestictions, you may have the data fro one area but not the neighbouring area.
Other boundaries may also be a factors, rivers, hills, airport, etc.
Or maybe some other local changes, cheap alcohol being made available where is might not have been before.

It’s a very worthwhile pursuit, and it must start with access to the data.

There our societies, so it’s our data, please give it to us when we ask.

Freedom of Information was supposed to push the door ajar and already that is being compromised left, right and centre.

If we take the analogy of ” no taxation without representation ” and decided to say in 2007 ” no taxation without access to what it is spent on ” maybe we could have another Boston tea party.

Would we run a pledge to say I will not pay my council tax if a million others would do the same. Could we get there as fact as the 1.8 million ( or thereabouts ) who went to Downing Street about road pricing ?

Of course, if you don’t have the data it can’t really do much…

by opening the data sets that drive government policy-making – which in the end is really about problem-solving – we will multiply society’s ability to analyze data, and vastly improve society’s ability to find innovative solutions to problems, big and small.

so keep pushing! … if it hasn’t happened yet, someone should organize a multi-disciplinary conference on this topic, to get a few groups mixing: free software/open project folk, government stats people, free data people, and then people expert in society-wide problems (say health, environment, energy, etc).

by mixing these groups we might be able to convince governments of the value to this movement; show the problem-solvers how groups of citizens might help; and help data-hounds and the open movement to understand where solutions might be built, if only we had more access to data.