In Annotating the Wild West of Information Flow I discussed a prototype of a ClaimReview-aware annotation client. ClaimReview is a standard way to format “a fact-checking review of claims made (or reported) in some creative work.” Now that both Google and Bing are ingesting ClaimReview markup, and fact-checking sites are feeding it to them, the workflow envisioned in that blog post has become more interesting. A fact checker ought to be able to highlight a reviewed claim in its original context, then capture that context and push it into the app that produces the ClaimReview markup embedded in fact-checking articles and slurped by search engines.
That workflow is one interesting use of annotation in the domain of fact-checking, it’s doable, and I plan to explore it. But here I’ll focus instead on using annotation to project claim reviews onto the statements they review, in original context. Why do that? Search engines may display fact-checking labels on search results, but nothing shows up when you land directly on an article that includes a reviewed claim. If the reviewed claim were annotated with the review, an annotation client could surface it.
To help make that idea concrete, I’ve been looking at ClaimReview markup in the wild. Not unexpectedly, it gets used in slightly different ways. I wrote a bookmarklet to help visualize the differences, and used it to find these five patterns:
{
"urlOfFactCheck": "https://www.washingtonpost.com/news/fact-checker/wp/2018/03/16/does-president-trumps-border-wall-pay-for-itself/?utm_term=.31281569ab46",
"reviewRating": {
"@type": "Rating",
"ratingValue": "-1",
"alternateName": "Fuzzy math",
"worstRating": "-1",
"bestRating": "-1",
"image": "https://dhpikd1t89arn.cloudfront.net/custom-rating/custom-ratings-b89013e8-729f-44aa-816e-93a9307b079c"
},
"itemReviewed": {
"@type": "CreativeWork",
"author": {
"@type": "Person",
"name": "Donald Trump ",
"jobTitle": "President",
"image": "https://dhpikd1t89arn.cloudfront.net/custom-rating/custom-ratings-b89013e8-729f-44aa-816e-93a9307b079c",
"sameAs": [
"www.whitehouse.gov"
]
},
"datePublished": "2018-03-13",
"name": "in a tweet"
},
"claimReviewed": "\"The $18 billion wall will pay for itself by curbing the importation of crime, drugs and illegal immigrants who tend to go on the federal dole.”"
}
Did a Captured Islamic State Leader Have Barack Obama On Speed Dial?
{
"urlOfFactCheck": "https://www.snopes.com/fact-check/did-captured-islamic-state-obama/",
"reviewRating": {
"alternateName": "False",
"ratingValue": "",
"bestRating": ""
},
"itemReviewed": {
"datePublished": "2018-03-13T08:25:46+00:00",
"name": "Snopes",
"sameAs": "http://dailyworldupdate.com/2018/03/11/breaking-captured-isis-leader-had-obama-on-speed-dial/"
},
"claimReviewed": "A captured Islamic State leader's cell phone contained the phone numbers of world leaders, including former President Barack Obama."
}
{
"urlOfFactCheck": "https://www.factcheck.org/2018/03/trump-misses-mars-attack/",
"reviewRating": {
"@type": "Rating",
"ratingValue": "-1",
"alternateName": "Clinton Endorsed Mars Flight",
"worstRating": "-1",
"bestRating": "-1",
"image": "https://dhpikd1t89arn.cloudfront.net/custom-rating/custom-ratings-76945159-07c1-49d6-833e-717c1da2478f"
},
"itemReviewed": {
"@type": "CreativeWork",
"author": {
"@type": "Person",
"name": "Donald Trump",
"jobTitle": "President of the United States",
"image": "https://dhpikd1t89arn.cloudfront.net/custom-rating/custom-ratings-76945159-07c1-49d6-833e-717c1da2478f",
"sameAs": [
"http://www.whitehouse.gov"
]
},
"datePublished": "2018-03-13",
"name": "Speech at military base"
},
"claimReviewed": "NASA “wouldn’t have been going to Mars if my opponent won.\""
}
{
"urlOfFactCheck": "http://www.politifact.com/texas/statements/2018/mar/15/ted-cruz/ted-cruz-says-beto-orourke-wants-open-borders-and-/",
"reviewRating": {
"@type": "Rating",
"ratingValue": "2",
"alternateName": "False",
"worstRating": "1",
"bestRating": "7",
"image": "https://dhpikd1t89arn.cloudfront.net/rating_images/politifact/tom-false.jpg"
},
"itemReviewed": {
"@type": "CreativeWork",
"author": {
"@type": "Person",
"name": "Ted Cruz",
"jobTitle": "U.S. senator for Texas",
"image": "https://dhpikd1t89arn.cloudfront.net/rating_images/politifatc/tom-false.jpg",
"sameAs": [
"https://www.youtube.com/watch?v=MvKElqdUxZg"
]
},
"datePublished": "2018-03-06",
"name": "Houston, Texas"
},
"claimReviewed": "Says Beto O’Rourke wants \"open borders and wants to take our guns.\""
}
Breitbart repeats blogger’s unsupported claim that NOAA manipulates data to exaggerate warming
{
"urlOfFactCheck": "https://climatefeedback.org/claimreview/breitbart-repeats-bloggers-unsupported-claim-noaa-manipulates-data-exaggerate-warming/",
"reviewRating": {
"@type": "Rating",
"alternateName": "Unsupported",
"bestRating": 5,
"ratingValue": 2,
"worstRating": 1
},
"itemReviewed": {
"@type": "CreativeWork",
"author": {
"@type": "Person",
"name": "James Delingpole"
},
"headline": "TBD",
"image": {
"@type": "ImageObject",
"height": "200px",
"url": "https://climatefeedback.org/wp-content/uploads/Sources/Delingpole_Breitbart_News.png",
"width": "200px"
},
"publisher": {
"@type": "Organization",
"logo": "Breitbart",
"name": "Breitbart"
},
"datePublished": "20 February 2018",
"url": "TBD"
},
"claimReviewed": "NOAA has adjusted past temperatures to look colder than they were and recent temperatures to look warmer than they were."
}
To normalize the difference I’ll need to look at more examples from these sites. But I’m also looking for more patterns, so if you know of other sites that routinely embed ClaimReview markup, please drop me links!
Would you say that most ClaimReviews you’ve come across are refutations? Perhaps worth considering where and when people are motivated to annotate and how this bias (if it exists) impacts the project of web annotation as a whole.
That’s a good question. Eyeballing https://www.realclearpolitics.com/fact_check_review/ I’d say that many are. But this has nothing to do with annotation per se. That’s the corpus of claims that have been reviewed by the leading fact-checking organizations.