When chatter in the mainstream media and in the blogosphere intersects with scientific discourse, I’m always interested in the ways that citations do, or don’t, cross the border between those domains. In 2006, for example, while checking references for a podcast with Steve Burbeck about multicellular computing, I traced a meme about how we humans are really a hybrid of human and bacterial cells. The mainstream vector was a New York Times magazine story on obesity. It got to the blogosophere by way of a Wired News story. But the original Nature Biotechnology article mentioned in the Wired story was linked nowhere that I could find.
A comment from Gordon Mohr on yesterday’s item about Many Eyes prompted a similar analysis. Gordon asks:
…do the Many Eyes founders consider the statistical paradox that when testing large numbers of hypotheses, *most* recognized ’statistically significant’ results may in fact be false?
A good discussion of the issue is here:
http://www.marginalrevolution.com/marginalrevolution/2005/09/why_most_publis.html
To answer Gordon’s question, I don’t know, it didn’t come up in our conversation. But lets look at the conversation surrounding the PloS Medicine article cited in the blog entry to which Gordon points.
The blog entry itself was widely noticed, it has 31 del.icio.us bookmarks. What about the PloS Medicine article cited in this popular blog entry? It has only 6 del.icio.us bookmarks.
This is the URL cited by the marginalrevolution blog:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=16060722
It’s not the most canonical form of the article’s URL. A more canonical form would be the base PubMed record:
http://www.ncbi.nlm.nih.gov/pubmed/16060722
That URL has 0 del.icio.us citations. However, now we cross over into the realm of scientific discourse. When you visit that PubMed URL, you’ll discover citations in the PubMed domain:
There’s another canonical form for the PloS Medicine article, by the way. It has a Digital Oject Identifier (DOI):
http://dx.doi.org/10.1371/journal.pmed.0020124
Interestingly, there is 1 del.icio.us citation for that DOI.
So, what did the PloS Medicine folks have to say about the claim in the cited August 2005 PloS Medicine article? Here’s an April 2007 reaction:
The mathematical proof offered for this in the PLoS Medicine paper shows merely that the more studies published on any subject, the higher the absolute number of false positive (and false negative) studies. It does not show what the papers’ graphs and text claim, viz, that the number of false claims will be a higher proportion of the total number of studies published (i.e., that the positive predictive value of each study decreases with increasing number of studies).
I’m not interested here in the claim and counterclaim. I’m interested in the process of discourse, in citation as the engine of that discourse, in the role that canonical identifiers play in citation, and in the disconnect between scientific and mainstream discourse.
It’s all happening on the web, but it’s happening in isolated ghettoes with few points of actual contact. How could we bring those worlds into closer contact?
Here’s one approach that could help. When the citation engines in the blogosphere find references in blog entries to scientific articles on the web, they could resolve those to their most canonical forms: DOIs, PubMed records. And they could make equivalences among those forms. That way, conversation in the blogosophere about a scientific article, and scientific conversation about the same article, would tend to hang together and would be discoverable in the same contexts.
Why does this matter? Well, the marginalrevolution blog is influential, widely cited in the blogosphere. The entry that cited the PLoS Medicine article was itself widely cited. But the PLoS Medicine reaction to the article is not part of the blog conversation. I had to work really hard to find it, and to include it here.
The conversation-tracking tools used by bloggers should discover scientific discourse related to a scientific article as easily as they discover blog discourse. Conversely, the conversation-tracking tools used by scientists should discover blog discourse as readily as scientific discourse. Public understanding of science would improve, and so would scientific understanding of the public.
Good luck; every time I try to bring up this topic (example, example 2) I get angry academics yelling at me. I absolutely cannot fathom why they don’t want to participate in the information commons.
I’m planning to write about it again soon, this is a good data point, thanks.
Jon
You should have been at the recent Scienceblogging conference (this past weekend). This very issue was on the table at the first session I attended. The problem remains how we mix “published” science with additional information. There are germs of efforts, e.g. Postgenomic, an aggregator and PLoS One, which supports trackbacks. But I think that’s not enough. Part of it is the need for a common entity to cluster around. Along similar lines as your post, I made my case for DOI’s to be available as a single URI for each publication around which this conversation can be built. I learnt later that the OpenRef project is trying to address this very conversation.
I have a feeling that some of these tools will mature/become more mainstream this year. Hopefully we will be able to capture the conversation, cause it’s a big problem right now.
“When the citation engines in the blogosphere find references in blog entries to scientific articles on the web, they could resolve those to their most canonical forms: DOIs, PubMed records.”
Postgenomic (conversation tracker), CiteULike and Connotea (both social bookmarking) all do this. I don’t think any of them track the “scientific conversation” yet, but they could do easily, the ‘comment’ data’s all in the PubMed API and the forward/backward citations are all in Scopus.
Is there a web service somewhere that will take a URL and return the equivalent canonical citation? If so, it wouldn’t be too hard to weld that into WordPress or some other popular blogging engine to automatically offer people who enter something else a chance to canonicalize on the spot.
I should add – by “they” I don’t mean all of them. Greg Wilson’s one of the academics I really admire, participating in the internet conversation. SPJ and too many others to name also do great work and let everyone in to read it.
“Postgenomie, CiteULike, Connotea”
So far as I can tell, none of these is set up to accept an URL as a query, like this one:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=16060722
Are there ways to do that?
“I absolutely cannot fathom why they don’t want to participate in the information commons. I’m planning to write about it again soon, this is a good data point, thanks.”
Here’s another for you. When I interviewed Timo Hannay…
http://itc.conversationsnetwork.org/shows/detail1864.html
…I mentioned that same first example about the human/bacteria hybrid. Even he had not realized that the NYT magazine article traced back to one of his own journals. As a scientific author or publisher, you’d have to want to know about those kinds of influences your work was having, wouldn’t you?
Jon:
http://www.postgenomic.com/api.php?type=posts&citing_paper=pmid:16060722
http://www.postgenomic.com/api.php?type=posts&citing_paper=doi:10.1371/journal.pmed.0020124
Greg:
http://www.connotea.org/webcite
eg
http://www.connotea.org/webcite?uri=http%3A%2F%2Fwww.pubmedcentral.nih.gov%2Farticlerender.fcgi%3Ftool%3Dpubmed%26pubmedid%3D16060722&fmt=json
Oh, and:
http://www.postgenomic.com/api.php?type=post&citing_url=http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=16060722
You would think so, but unfortunately that is not the case. Scientists are much too conservative and locked up in their own cliques. It also explains the huge disconnect between science and the general public. Science, esp Life Science, has become so much more complex in recent years, but we’ve failed to open up the communication channels. It’s almost as if the approach is “this is complex stuff, so we’ll keep it to ourselves”.
The good news is that slowly, but surely, thanks in parts to efforts from Nature and PLoS, and people like Alf (who commented above) and Euan Adie, we have a movement towards making this more of a conversation around science, and not just one limited to a publication
Why does it have to be so hard? Why can’t we just have citations in the form of
http://yourfavoritecitationresolver.com/journalname/volume/issue/pagenumber?
In one stroke you’ve got a link which resolves to one unique paper and a URL which is resistant to link rot, because you can just change the domain if yourfavoritecitationresolver.com goes down.
The &fmt=json is a nice touch, alf.
I think a lot of this is the nature of how people get news. ScienceDaily, for example, doesn’t use accurate citations because AAAS doesn’t publish them in the Eurekalert newsfeeds that a lot of press release like that use. If they do, it is below the ### and a lot of those are automatic scripts for formatting so they strip it out.
I think most individual scientific bloggers cite pretty accurately. However, a PLoS article is not really written for the public so they are going to be rewritten by science publications for general readers. Journalists in mainstream publications don’t write lists of sources in articles and never have.
If people come up with an agreed standard, and in turn can find things through postgenomic or anyone else, others will use it. However, the URL I am on has a million articles read this month and doesn’t show up in postgenomic anywhere at all so they would probably need to include more than a few blogs if they’re going to be used by the masses in science writing.
I think Mr. Gunn has a pretty good solution.
Isn’t another strategy to connect these communities to encourage scientists to blog?
Sorry I’m late to the game here. My first two attempts to post seem to have been lost in the ether. Sunspots, undoubtedly.
Anyway- this certainly doesn’t address all of the issues raised here, but this past weekend, my CrossRef colleague, Amy Brand, was at the Science Blogging Conference where she showed a WordPress plug-in that we are creating (Movable Type soon to follow) that will allow bloggers to more easily cite formal scholarly (not just scientific) publications in a consistent (dare I say “canonical?”) way. The plug-in will allow the blog author to search the CrossRef metadata for references and then insert a formatted and DOI-linked citation directly into their blog posting. For good measure- the citation will also be backed by COinS metadata so that metadata-aware apps like Zotero, CiteUlike and Connotea can pick them up. Virtuous circle and all that.
Again- clearly this plug-in might be just a wee step in addressing the whole host of issues surrounding the interaction between formal/informal online scholarly discourse, but we are hoping that, if we at least make using DOIs a little easier, they’ll gain some more uptake in the academic blogging community.
We will post on the CrossTech blog when the plug-in is ready for distribution.
And Greg’s idea of a url2doi reverse lookup service is very interesting. We need to think about that.
You would think so, but unfortunately that is not the case. Scientists are much too conservative and locked up in their own cliques.
That’s one side of it; the other is that bloggers are utterly consumed with their own cliques and wildly exaggerate the degree to which scientists should be interested in them. They also have an absurdly distorted view of science, in which their pet researchers are all major players, most research effort goes into evolution and climate change research, and the vast body of other scientists and fields barely exists.
Bottom line: as of today, there’s little for scientists to gain, by the standards by which nearly all of them would define “gain”.
Reading something over at ScienceBlogs, I see my point being made rather vividly:
http://scienceblogs.com/pharyngula/2008/01/i_endorse_francis_collins_for.php?utm_source=mostactive&utm_medium=link
Now, this appeared in the TOP FIVE sidebar, it’s by probably the most prominent life science blogger, one who is apparently some sort of professor! Honestly, if I were to show my colleagues that post and the one it links to (“I think Collins has no academic credibility.”) they’d split their sides laughing. If I suggested they distribute their results in order to get more exposure among a bunch of retards like that, they’d have me committed.
“Bottom line: as of today, there’s little for scientists to gain, by the standards by which nearly all of them would define “gain”.”
Agreed. Improving public understanding of science isn’t part of the job description, nor is improving scientific understanding of the public. So we wind up relying on the media (and now, increasingly, the blogosphere) as mediators.
“That’s one side of it; the other is that bloggers are utterly consumed with their own cliques and wildly exaggerate the degree to which scientists should be interested in them.”
Fair point. Note that in the example I gave, though, I’m not suggesting that scientists directly monitor and try to actively influence the blog discourse (though sometimes maybe they should).
I’m only suggesting that when blog discourse touches on primary scientific literature, the discourse that’s happening in the scientific context should be directly available in the blog context. So that, for example, when the blogosphere reacts to a PLoS Medicine article, it will tend to be more aware of the scientific reaction to that same article.
In this scenario the scientific authors wouldn’t need to do anything more or different. It just that their existing process would be more fully and coherently available.
Agreed. Improving public understanding of science isn’t part of the job description, nor is improving scientific understanding of the public.
Actually, I think most researchers would disagree with you on that; although not that many of them ever actually do anything about it. However, science bloggers and blog readers are a tiny fraction of “the public”, and most of them are more interested in dragging science into their feverish political views than in science itself.
I’m only suggesting that when blog discourse touches on primary scientific literature, the discourse that’s happening in the scientific context should be directly available in the blog context.
OK, to the degree that you’re talking about scientists not having to do anything, fine. (Although with cases like the one I linked earlier, I suspect a lot of researchers would actively prefer to avoid being even indirect linked to such a forum.)
I’d also note that responses and counter-responses are a relatively marginal part of “the discourse that’s happening in the scientific context”. I don’t believe I’ve ever had any on anything on which I’ve been a primary author. Bloggers have this odd reverence for peer-reviewed publications (not that responses are peer-reviewed anyway) but most discussion happens at seminars and in review articles. But as you say, if it can be done for free, it can only be a good thing.
I’m a biologist and a postdoc at Duke University.
There are at least two major impediments to the integration of scientific discourse into the information commons (as commenter 1 called it):
1. Most published scientific discourse occurs in journals that are barricaded behind paywalls. BMC, PLoS, and their ilk are welcome exceptions, but they are indeed exceptions. In my day-to-day work, most of the references I consult are accessible to me only because my institution has bought subscriptions, often for obscene sums of money. This includes all the most highly cited nonspecialist journals in which biologists regularly publish (Current Biology, Nature, Science, etc.) except PLoS Biology. The situation is a scandal for many reasons, not least that almost all the research published in these journals is paid for by grants from government agencies; it’s your tax dollars at work, but you can’t read it.
2. Academia generally doesn’t reward scientists for publishing anywhere else. At research universities, faculty members are hired, promoted, or fired largely according to their success in getting grants and publishing articles in such journals. So-called outreach is nice but unnecessary. Few faculty members, especially junior ones, feel they can afford to spend much time on such optional activities.
(Further to 1, it’s true that many of the contracts we sign with journals publishing our work allow us to post copies on our web sites. But even if we all did so, having access to all the articles in an issue of a journal via several dozen web sites would be inferior to having access to the issue itself. Aggregation matters, and aggregation of scientific discourse across the web must become far more effective in order to pose a serious challenge to the commercial publishers of scientific journals.)
“it’s your tax dollars at work, but you can’t read it”
There are hopeful signs on that front:
http://www.taxpayeraccess.org/media/release07-1226.html
Though as John Faughnan wryly observes:
http://jfaughnan.blogspot.com/2007/12/happy-solstice-quiet-revolution-in.html
“I wonder if Bush knew what he signed. The open access provision would have been buried deeply in the bill.”
sorry to change the subject, but what are all of your ideas about the worlds largest atom smasher and how it will act on a subatomic level? will it create black holes? and if it does can we save our selves with another balck hole? what might it do in theory? if you could email me at gtasacd2yahoo.com
After reading http://blog.jonudell.net/2008/01/22/bloggers-talk-to-bloggers-scientists-talk-to-scientists/, I thought this might be a useful resource for your site:
http://www.biomedsearch.com
The site is free, and perhaps the most comprehensive biomedical site on the web. It has all PubMed and MedLine documents, plus mililons more including full-text journal articles and a large database of theses and dissertations.
And, you don’t have to register but if you do you can use portfolios to save documents, share documents (and comment on them) between users, and set up automatic alerts.
Thank you a lot for sharing this with all of us you actually know what you’re talking approximately! Bookmarked. Kindly also visit my website =). We can have a hyperlink change contract between us