The Woodard projection

In a memorable episode of The West Wing, visitors from the Cartographers for Social Justice upend CJ’s and Josh’s worldviews.

Cartographer: “The Peters projection.”

CJ: “What the hell is that?”

Cartographer: “It’s where you’ve been living this whole time.”

I’m having the same reaction to Colin Woodard’s 2011 book American Nations. He sees North America as three federations of nations. The federation we call the United States comprises nations he calls Yankeedom, New Netherland, The Midlands, Tidewater, Greater Appalachia, The Deep South, El Norte, The Far West, and The Left Coast.

Here’s his definition of a nation:

A nation is a group of people who share — or believe they share — a common culture, ethnic origin, language, historical experience, artifacts, and symbols.”

Worldwide some nations are stateless, some align with state borders, and some cut across state borders. North America’s eleven nations are, in his view, stateless, cutting across state boundaries in ways I find disorienting but enlightening.

On his map (from a 2013 Tufts alumni magazine article now readable only at the Internet Archive) I have marked the US places where I’ve lived.

Until now, if you asked me where I’ve lived, I’d have said East Coast and Midwest and recently California. According to the Woodard projection I have lived in four nations: Yankeedom, The Midlands, Tidewater, and The Left Coast. It wasn’t easy to locate my homes on his map. They all occupy a narrow band of latitude. On the East Coast, that band touches three of Woodard’s nations. In two of those, Yankeedom and The Midlands, I lived near the cradles of nations that spread far north and west.

I’m from near Philadelphia, in The Midlands, “founded by English Quakers, who welcomed people of many nations and creeds to their utopian colonies on the shores of Delaware Bay.” That resonates. I grew up in a place called Plymouth Meeting and went to Quaker kindergarten there. It would never have occurred to me that Philadelphia is culturally closer to places as far west as Nebraska, and as far north as the province of Ontario, than to Baltimore or Boston. Likewise I never thought of Ann Arbor, where I called myself a midwesterner, as part of a culture that flowed west from Boston. Or that Baltimore sits at the intersection of three nations.

These eleven nations have been hiding in plain sight throughout our history. You see them outlined on linguists’ dialect maps, cultural anthropologists’ maps of material culture regions, cultural geographers’ maps of religious regions, campaign strategists’ maps of political geography, and historians’ maps of the pattern of settlement across the continent.

Two of the eleven nations, Yankeedom and The Deep South, have been “locked in nearly perpetual combat for control of the federal government since the moment such a thing existed,” Woodard says.

The analysis, which builds on prior art that he cites, may be a helpful way to contextualize the 2016 US election.

“The Woodard projection.”

“What the hell is that?”

“It’s where you’ve been living this whole time.”

Don’t just Google it! First, let’s talk!

Asking questions in conversation has become problematic. For example, try saying this out loud: “I wonder when Martin Luther King was born?” If you ask that online, a likely response is: “Just Google it!” Maybe with a snarky link: http://lmgtfy.com/?q=when was martin luther king born?

Asking in conversation is frowned upon too. Why ask us when you could ask Siri or Alexa to ask Google?

There’s a kind of shaming going on here. We are augmented with superpowers, people imply, you’re an idiot to not use them.

Equally problematic is the way this attitude shuts down conversation. Sure, I can look up MLK’s birthday. Or you can. But what if our phones are dead? Now we need to rely on our own stored knowledge and, more importantly, on our collective pool of stored knowledge.

I think MLK was shot in 1968. I’d have been twelve. Does that sound right? Yeah, we were in the new house, it was sixth grade. And I know he died young, maybe 42? So I’ll guess 1968 – 42 = 1926.

Were you around then? If so, how do you remember MLK’s assassination? If not, what do you know about the event and its context?

As you can see in the snarky screencast, I’m wrong, MLK died even younger than I thought. You might know that. If we put our heads together, we might get the right answer.

Asking about something that can easily be looked up shouldn’t stop a conversation, it should start one. Of course we can look up the answer. But let’s dwell for a minute on what we know, or think we know. Let’s exercise our own powers of recall and synthesis.

Like all forms of exercise, this one can be hard to motivate, even though it’s almost always rewarding. Given a choice between an escalator and the stairs, I have to push myself to prefer the stairs. In low-stakes wayfinding I have to push myself to test my own sense of direction. When tuning my guitar I have to push myself to hear the tones before I check them. I am grateful to be augmented by escalators, GPS wayfinders, and tuners. But I want to use these powers in ways that complement my own. Finding the right balance is a struggle.

Why bother? Consider MLK. In The unaugmented mind I quoted from MLK confidante Clarence Jones’ Behind the Dream:

What amazed me was that there was absolutely no reference material for Martin to draw upon. There he was [in the Birmingham jail] pulling quote after quote from thin air. The Bible, yes, as might be expected from a Baptist minister, but also British prime minister William Gladstone, Mahatma Gandhi, William Shakespeare, and St. Augustine.

And:

The “Letter from a Birmingham Jail” showed his recall for the written material of others; his gruelling schedule of speeches illuminated his ability to do the same for his own words. Martin could remember exact phrases from several of his unrelated speeches and discover a new way of linking them together as if they were all parts of a singular ever-evolving speech. And he could do it on the fly.

Jones suggests that MLK’s power flowed in part from his ability to reconfigure a well-stocked working memory. We owe it to ourselves, and to one another, to nurture that power. Of course we can look stuff up. But let’s not always go there first. Let’s take a minute to exercise our working memories and have a conversation about them, then consult the oracle. I’ll bet that in many cases that conversation will turn out to be the most valuable part of the experience.

My new library superpower

The recommendations that matter to me — for books to read, movies to watch, products to buy, places to visit — almost never come from algorithms. Instead they come from friends, family, acquaintances, and now also, in the case of books, from fellow library patrons whom I’ve never met.

This last source is a recent discovery. Here’s how it works. When I enter the library, I walk past shelves of new books and staff picks. These are sometimes appealing, but there’s a much better source of recommendations to be found at the back of the library. There, on shelves and carts, are the recently-returned books ready to be reshelved. I always find interesting titles among them. Somehow I had never noticed that you can scan these titles, and check out the ones you want before they ever get reshelved. Did it always work this way at our library? At libraries in other places I’ve lived? If so I am woefully late to the party, but for me, at least, this has become a new superpower.

I reckon that most books in the library rarely if ever leave the shelves. The recently-returned section is a filter that selects for books that patrons found interesting enough to check out, read (or maybe not), and return. That filter has no bias with respect to newness or appeal to library staff. And it’s not being manipulated by any algorithm. It’s a pure representation of what our library’s patrons collectively find interesting.

The library’s obvious advantage over the bookstore is the zero price tag. But there’s a subtler advantage. In the bookstore, you can’t peruse an anonymized cache of recently-bought books. Online you can of course tap into the global zeitgeist but there you’re usually being manipulated by algorithms. LibraryThing doesn’t do that, and in principle it’s my kind of place online, but in practice I’ve never succeeded in making it a habit. I think that’s because I really like scanning titles that I can just take off a shelf and read for free. Not even drone delivery can shorten the distance between noticing and reading to seconds. Ebooks can, of course. But that means another screen, I already spend too much time looking at screens, the print medium provides a welcome alternative.

This method has been so effective that I’ve been (guiltily) a bit reluctant to describe it. After all, if everyone realized it’s better to bypass the new-book and staff-pick displays and head straight for the recently-returned area, the pickings would be slimmer there. But I’m nevertheless compelled to disclose the secret. Since few if any of my library’s patrons will ever read this post, I don’t think I’m diluting my new superpower by explaining it here. And who knows, maybe I’ll meet some interesting people as I exercise it.

 

Designing for least knowledge

In a post on the company blog I announced that it’s now possible to use the Hypothesis extension in the Brave browser. That’s great news for Hypothesis. It’s awkward, after all, to be an open-source non-profit software company whose product is “best viewed in” Chrome. A Firefox/Hypothesis extension has been in the works for quite a while, but the devil’s in the details. Although Firefox supports WebExtensions, our extension has to be ported to Firefox, it isn’t plug-and-play. Thanks to Brave’s new ability to install unmodified Chrome extensions directly from the Chrome web store, Hypothesis can now plug into a powerful alternative browser. That’s a huge win for Hypothesis, and also for me personally since I’m no longer tethered to Chrome in my daily work.

Another powerful alternative is forthcoming. Microsoft’s successor to Edge will (like Brave) be built on Chromium, the open-source engine of Google’s Chrome. The demise of EdgeHTML represents a loss of genetic diversity, and that’s unfortunate. Still, it’s a huge investment to maintain an independent implementation of a browser engine, and I think Microsoft made the right pragmatic call. I’m now hoping that the Chromium ecosystem will support speciation at a higher level in the stack. Ever since my first experiments with Greasemonkey I’ve been wowed by what browser extensions can do, and eager for standardization to unlock that potential. It feels like we may finally be getting there.

Brave’s support for Chrome extensions matters to me because I work on a Chrome extension. But Brave’s approach to privacy matters to me for deeper reasons. In a 2003 InfoWorld article I discussed Peter Wayner’s seminal book Translucent Databases, which explores ways to build information systems that work without requiring the operators to have full access to users’ data. The recipes in the book point to a design principle of least knowledge.

Surveillance capitalism knows way too much about us. Is that a necessary tradeoff for the powerful conveniences it delivers? It’s easy to assume so, but we haven’t really tried serious alternatives yet. That’s why this tweet made my day. “We ourselves do not know user identities or how donations might link via user,” wrote Brendan Eich, Brave’s founder. “We don’t want to know.”

We don’t want to know!

That’s the principle of least knowledge in action. Brave is deploying it in service of a mission to detoxify the relationship between web content and advertising. Will the proposed mechanisms work? We’ll see. If you’re curious, I recommend Brendan Eich’s interview with Eric Knorr. The first half of the interview is a deep dive into JavaScript, the second half unpacks Brave’s business model. However that model turns out, I’m grateful to see a real test of the principle. We need examples of publishing and social media services that succeed not as adversaries who monetize our data but rather as providers who deliver convenience at a reasonable price we’re happy to pay.

My hunch is that we’ll find ways to build profitable least-knowledge services once we start to really try. Successful examples will provide a carrot, but there’s also a stick. Surveillance data is toxic stuff, risky to monetize because it always spills. It’s a liability that regulators — and perhaps also insurers — will increasingly make explicit.

Don’t be evil? How about can’t be evil? That’s a design principle worth exploring.

Highlighting passages doesn’t aid my memory, but speaking them does

When I was in college, taking notes on textbooks and course readings, I often copied key passages into a notebook. There weren’t computers then, so like a medieval scribe I wrote out my selections longhand. Sometimes I added my own notes, sometimes not, but I never highlighted, even in books that I owned. Writing out the selections was a way to perform the work I was reading, record selections in my memory, and gain deeper access to the mind of the author.

Now we have computers, and the annotation software I help build at Hypothesis is ideal for personal note-taking. Close to half of all Hypothesis annotations are private notes, so clearly lots of people use it that way. For me, though, web annotation isn’t a private activity. I bookmark and tag web resources that I want to keep track of, and collaborate with others on document review, but I don’t use web annotation to enhance my private reading.

To be sure, I mostly read books and magazines in print. It’s a welcome alternative to the screens that otherwise dominate my life. But even when my private reading happens online, I don’t find myself using our annotation tool the way so many others do.

So, what’s a good way to mark and remember a passage in a book if you don’t want to highlight it, or in the case of a library book, can’t highlight it? I thought about the scribing I used to do in college, and realized there’s now another way to do that. Recently, when I read a passage in a book or magazine that I want to remember and contemplate, I’ve been dictating it into a note-taking app on my phone.

I’ve followed the evolution of speech-to-text technology with great interest over the years. When I reviewed Dragon NaturallySpeaking, I did what every reviewer does. I tried to use the tool to dictate my review, and got mixed results. Over time the tech improved but I haven’t yet adopted dictation for normal work. At some point I decided to forget about dictation software until it became something that civilians who weren’t early-adopter tech journos used in real life.

One day, when I received some odd text messages from Luann, I realized that time had arrived. She’d found the dictation feature on her phone. It wasn’t working perfectly, and the glitches were amusing, but she was using it in an easy and natural way, and the results were good enough.

I still don’t dictate to my computer. This essay is coming to you by way of a keyboard. But I dictate to my phone a lot, mostly for text messages. The experience keeps improving, and now this new practice — voicing passages that I read in books, in order to capture and remember them — seems to be taking hold.

I’m reminded of a segment in a talk given by Robert “R0ml” Lefkowitz at the 2004 Open Source Conference, entitled The Semasiology of Open Source (part 2), the second in a series structured as thesis (part 1), antithesis (part 2), and synthesis (part 3). ITConversation aptly described this luminous series of talks as “an intellectual joy-ride”; I’m going to revisit the whole thing on a hike later today.

Meanwhile, here’s a transcription of the segment I’m recalling. It appears during a review of the history of literacy. At this point we have arrived at 600 AD.

To be a reader was not to be the receiver of information, it was to be the transmitter of the information, because it was not possible to read silently. So things that were written were written as memory aids to the speaker. And the speaker would say the words to the listener. To read was to speak, and those were synonyms … The writing just lies there, whereas the speaking lifts it off the page. The writing is just there, but the speaking is what elevates the listener.

Had I merely read that passage I’m certain I would not remember it 14 years later. Hearing R0ml speak the words made an indelible impression. (Seeing him speak the words, of course, made it even more indelible.)

Silent reading, once thought impossible, had to be invented. But just because we can read silently doesn’t mean we always should, as everyone who’s read aloud to a young child, or to a vision-impaired elder, knows. It’s delightful that voice recognition affords new ways to benefit from the ancient practice of reading aloud.

A small blog neighborhood hiding in plain sight

For as long as I can remember, I’ve tweeted every blog post. As an experiment, I didn’t do that with this one. As a result, WordPress tells me that relatively few people have read it. But I’m not monetizing pageview counters here, why do I care? The interaction with those who did read that item was pleasant.

So, the experiment continues. Some items here, like this one, can enjoy a more intimate space than others, simply by not announcing themselves on Twitter. A small blog neighborhood hiding in plain sight.

Where’s my Net dashboard?

Yesterday Luann was reading a colleague’s blog and noticed a bug. When she clicked the Subscribe link, the browser loaded a page of what looked like computer code. She asked, quite reasonably: “What’s wrong? Who do I report this to?”

That page of code is an RSS feed. It works the same way as the one on her own blog. The behavior isn’t a bug, it’s a lost tradition. Luann has been an active blogger for many years, and once used an RSS reader, but for her and so many others, the idea of a common reader for the web has faded.

There was a time when most of the sources I cared about flowed into such a reader: mainstream news, a vibrant and growing blogosphere, podcasts, status updates, standing search queries, you name it. The unread item count could get daunting, but I was able to effectively follow a wide assortment of information flows in what I came to think of as my Net dashboard.

Where’s my next dashboard? I imagine a next-gen reader that brings me the open web and my social circles in a way that helps me attend to and manage all the flow. There are apps for that, a nice example being FlowReader, which has been around since 2013. I try these things hopefully but so far none has stuck.

Information overload, once called infoglut, remains a challenge. We’re all flooded with more channels than we can handle, more conversations happening in more places than we can keep track of.

Fear of missing out (FOMO) is the flip side of infoglut. We expect that we should be able to sanely monitor more than we actually can.

The first-gen reader didn’t solve infoglut/FOMO, nothing could, but for a while, for me, it was better than the alternative, which was (and now is again) email. Of course that was me, a tech journalist who participated in, researched, and wrote about topics in Net technology and culture — including RSS, which animated the dashboard I used to keep track of everything else. It was, however, a workflow that researchers and analysts in other fields will recognize.

Were I were doing the same kind of work today, I’d cobble together the same kind of dashboard, while grumbling about the poorer experience now available. Instead my professional information diet is narrower and deeper than when analytical writing for commercial audiences was my work. My personal information diet, meanwhile, remains as diverse as everyone’s.

So I’m not sure that a next-gen reader can solve the same problems that my first-gen reader did, in the same ways. Still, I can’t help but envision a dashboard that subscribes to, and manages notifications from, all my sources. It seems wrong that the closest thing to that, once more, is email. Plugging the social silos into a common reader seems like the obvious thing. But if that were effective, we’d all be using FlowReader or something like it.

Why don’t we? Obviously the silos can refuse to cooperate, as FlowReader noted when announcing the demise of its Facebook integration:

These changes were made [by Facebook] to give users more control over their own data, which we support. It’s a great thing for users! However, it also means that Facebook data is no longer going to be easy to share between applications.

You know what would be a really great thing for users, though? A common reader that makes it easy to keep track of friends and family and coworkers along with news and all kinds of personal and professional information sources.

“What’s wrong?”

It’s not just that the silos can shut down their feeds. It’s that we allowed ourselves to get herded into them in the first place. For a while, quite a few people got comfortable with the notion of publishing and subscribing to diverse feeds in a common way, using systems that put them in charge of outflow and inflow. In one form or another that’s still the right model. Sometimes we forget things and have to relearn them. This is one of those things.

“Who do I report this to?”

Everyone.

The reengineering of three California lakes

If you drive up the the eastern side of California, you’ll encounter three ancient lakebeds transformed by human engineering during the last century. The sequence goes like this: Salton Sea, Owens Valley, Mono Lake. We visited all three on a recent road trip. Since returning I’ve learned that their stories continue to unfold.

The Salton Sea

The Salton Sea, almost twice the size of Lake Tahoe, was created accidentally in 1905 when an irrigation project went out of control and, for a year and a half, sucked the Colorado river into what had been a dry lakebed. As a recent immigrant to California I can confirm that many natives don’t know much, if anything, about this place. A 2004 documentary narrated by John Waters, Plagues & Pleasures on the Salton Sea, traces its arc through a boom era of sport fishing and real estate speculation to what is now a living ghost town. For decades people have talked about saving the Salton Sea. That may once have meant restoring California’s “lost Riviera” but nowadays it’s all about mitigating an environmental disaster.

The Salton Sink is a low-lying basin that has flooded and dried many times over hundreds of thousands of years. What makes the current drying phase different is that the only inflow now is agricultural runoff with high concentrations of pesticides. As the lake evaporates, toxic dust goes windborne, threatening not only the Salton Sea communities, but also nearby Mexicali, Mexico, a metropolitan area with a million people. Meanwhile the lake’s increasing salinity is ruining the habitat of millions of migratory birds. These looming health and environmental crises motivated California to allocate $200 million as part of a successful June 2018 ballot initiative (Prop. 68) to stabilize the Salton Sea. (Another $200 million won’t be forthcoming because it was part of a failed follow-on ballot initiative in November 2018, Prop. 3.)

In an effort to buy time, a 2003 agreement to transfer water from the Imperial Valley to San Diego required the release of “mitigation water” to the Salton Sea. That ended in 2017 with no clear roadmap in place. What would it mean to stabilize the Salton Sea? A dwindling Colorado River won’t be the answer. It may be possible to import seawater from the Pacific, or the Gulf of California, to create a “smaller but sustainable” future that balances the needs of the region’s people and wildlife in a context of growing crises of drought and fire. Dry methods of dust suppression might also help. But all methods will require major engineering of one kind or another. The cost of that 1905 accident continues to grow.

The Owens Valley

Further north lies Owens Lake, mostly drained since 1913 when William Mulholland began siphoning its water into the LA aqueduct. The movie Chinatown is a fictionalized account of the battle for that water. What remains is mostly a vast salt flat that’s patrolled by mining vehicles and was, until recently, the nation’s worst source of dust pollution. From highway 395 it still looks like the surface of some other planet. But mitigation began at the turn of this century, and by 2017 LA’s Department of Water and Power had spent nearly $2 billion on what a modern western explorer known as StrayngerRanger describes as “a patchwork of dust-smothering techniques, including gravel, flooded ponds and rows of planted vegetation that cover nearly 50 square miles — an area more than twice the size of Manhattan.”

A key innovation, according to the LA Times, “involves using tractors to turn moist lake bed clay into furrows and basketball-sized clods of dirt. The clods will bottle up the dust for years before breaking down, at which point the process will be repeated.” This dry method was a big success, the dust has settled, public trails opened in 2016, and as Robin Black’s photography shows, life is returning to what’s left of the lake.

The LA Times:

In what is now hailed as an astonishing environmental success, nature quickly responded. First to appear on the thin sheen of water tinged bright green, red and orange by algae and bacteria were brine flies. Then came masses of waterfowl and shorebirds that feed on the insects.

Robin Black:

This was never supposed to happen, and it’s a BIG DEAL. This, the habitat creation and management. This, the welcoming of the public to the lake. This is a huge victory for the groups in the Owens Valley who fought so tirelessly to make this happen.

We need to celebrate environmental successes and learn from them. And on this trip, the best one was yet to come.

Mono Lake

Mono Lake, near the town of Lee Vining (“the gateway to Yosemite”), was the next object of LA’s thirst. Its inflow was diverted to the aqueduct, and the lake almost went dry. Now it’s filling up again, thanks to the dedicated activists who formed the Mono Lake Committee 40 years ago, sued the LA Department of Water and Power, won back 3/4 of the inflow, and as chronicled in their video documentary The Mono Lake Story, have been stewarding the lake’s recovery ever since.

Today you can walk along the newly-restored Lee Vining Creek trail and feel the place coming alive again. There’s been cleanup and trail-building and planting but, for the most part, all that was needed was to let the water flow again.

How did LA compensate for the lost flow? This 30-second segment of the documentary answers the question. The agreement included funding that enabled LA to build a wastewater reclamation plant to make up the difference. It wasn’t a zero-sum game after all. There was a way to maintain a reliable water supply for LA and to restore Mono Lake. It just took people with the vision to see that possibility, and the will to make it real.

Renaming Hypothesis tags

Wherever social tagging is supported as an optional feature, its use obeys a power law. Some people use tags consistently, some sporadically, most never. This chart of Hypothesis usage illustrates the familiar long-tail distribution:

Those of us in the small minority of consistent taggers care a lot about the tag namespaces we’re creating. We tag in order to classify resources, we want to be able to classify them consistently, but we also want to morph our tag namespaces to reflect our own changing intuitions about how to classify and also to adapt to evolving social conventions.

Consistent tagging requires a way to make, use, and perhaps share a list of controlled tags, and that’s a topic for another post. Morphing tag namespaces to satisfy personal needs, or adapt to social conventions, requires the ability to rename tags, and that’s the focus of this post.

There’s nothing new about the underlying principles. When I first got into social tagging, with del.icio.us back in the day, I made a screencast to show how I was using del.icio.us’ tag renaming feature to reorganize my own classification system, and to evolve it in response to community tag usage. The screencast argues that social taggers form a language community in which metadata vocabularies can evolve in the same way natural languages do.

Over the years I’ve built tag renamers for other systems, and now I’ve made one for Hypothesis as shown in this 90-second demo. If you’re among the minority who want to manage your tags in this way, you’re welcome to use the tool, here’s the link. But please proceed with care. When you reorganize a tag namespace, it’s possible to wind up on the wrong end of the arrow of entropy!

Letters to Mr. Wilson’s Museum of Jurassic Technology

Dear Mr. Wilson,

Your Museum of Jurassic Technology (MJT) first came to my attention in 1995 when I read an excerpt from Lawrence Weschler’s Mr. Wilson’s Cabinet of Wonder in the New Yorker. Or anyway, that’s the origin story I’ve long remembered and told. As befits the reality-warping ethos of the MJT, it seems not to be true. I can find no record of such an article in the New Yorker’s archive. (Maybe I read it in Harpers?) What I can find, in the New York Times’ archive, is a review of the book that nicely captures that ethos:

Run by its eccentric proprietor out of a storefront in Culver City, Calif., the museum is clearly a modern-day version, as Mr. Weschler astutely points out, of the “wonder-cabinets” that sprang up in late Renaissance Europe, inspired by all the discoveries in the New World. David Wilson comes off as an amusingly Casaubonesque figure who, in his own little way, seeks to amass all the various kinds of knowledge in the world; and if his efforts seem random and arcane, they at any rate sound scientifically specific. Yet when Mr. Weschler begins to check out some of the information in the exhibits, we discover that much of it is made up or imagined or so elaborately embroidered as to cease to resemble any real-world facts.

The key to the pleasure of this book lies in that “much of,” for the point of David Wilson’s museum is that you can’t tell which parts are true and which invented. In fact, some of the unlikeliest items — the horned stink ants, for instance — turn out to be pretty much true. In the wake of its moment of climactic exposure, “Mr. Wilson’s Cabinet of Wonder” turns into an expedition in which Lawrence Weschler tracks down the overlaps, correspondences and occasionally tenuous connections between historical and scientific reality on the one hand and the Museum of Jurassic Technology on the other.

We’ve always wanted to visit the MJT, and finally made the pilgrimage a few weeks ago. Cruising down the California coast on a road trip, the museum was our only LA destination. The miserable traffic we fought all day, both entering and leaving Culver City, was a high price to pay for several hours of joy at the museum. But it was worth it.

At one point during our visit, I was inspecting a display case filled with a collection of antique lace, each accompanied by a small freestanding sign describing the item. On one of the signs, the text degrades hilariously. As I remember it (no doubt imperfectly) there were multiple modes of failure: bad kerning, faulty line spacing, character misencoding, dropouts, faded print. Did I miss any?

While I was laughing at the joke, I became aware of another presence in the dark room, a man with white hair who seemed to pause near me before wandering off to an adjoining dark room. I thought it might be you, but I was too intimidated to ask.

The other night, looking for videos that might help me figure out if that really was you, I found a recording of an event at the USC Fisher Museum of Art. In that conversation you note that laughter is a frequent response to the MJT:

“We don’t have a clue what people are laughing about,” you said, “but it’s really hard to argue with laughter.”

I was laughing at that bit of hilariously degraded signage. I hope that was you hovering behind me, and I hope that you were appreciating my appreciation of the joke. But maybe not. Maybe that’s going to be another questionable memory. Whatever the truth may be, thank you. We need the laughs now more than ever.

Yours sincerely,

Jon Udell

Searching across silos, circa 2018

The other day, while searching across various information silos — WordPress, Slack, readthedocs.io, GitHub, Google Drive, Google Groups, Zendesk, Stack Overflow for Teams — I remembered the first time I simplified that chore. It was a fall day in 1996. I’d been thinking about software components since writing BYTE’s Componentware1 cover story in 1994, and about web software since launching the BYTE website in 1995. On that day in 1996 I put the two ideas together for the first time, and wrote up my findings in an installment of my monthly column entitled On-Line Componentware. (Yes, we really did write “On-Line” that way.)

The solution described in that column was motivated by McGraw-Hill’s need to unify search across BYTE and some other MgH pubs that were coming online. Alta Vista, then the dominant search engine, was indexing all the sites. It didn’t offer a way to search across them. But it did offer a site: query as Google and Bing do today, so you could run per-site queries. I saw it was possible to regard Alta Vista as a new kind of software component for a new kind of web-based application that called the search engine once per site and stitched the results into a single web page.

Over the years I’ve done a few of these metasearch apps. In the pre-API era that meant normalizing results from scraped web pages. In the API era it often means normalizing results fetched from per-site APIs. But that kind of normalization was overkill this time around, I just needed an easier way to search for words that might be on our blog, or in our docs, or in our GitHub repos, or in our Google storage. So I made a web page that accepts a search term, runs a bunch of site-specific searches, and opens each into a new tab.

This solution isn’t nearly as nice as some of my prior metasearchers. But it’s way easier than authenticating to APIs and merging their results, or using a bunch of different query syntaxes interactively. And it’s really helping me find stuff scattered across our silos.

But — there’s always a but — you’ll notice that Slack and Zendesk aren’t yet in the mix. All the other services make it possible to form a URL that includes the search term. That’s just basic web thinking. A set of web search results is an important kind of web resource that, like any other, should be URL-accessible. Unless I’m wrong, in which case someone please correct me, I can’t do the equivalent of `?q=notifications` in a Slack or Zendesk URL. Of course it’s possible to wrap their APIs in a way that enables URL query, but I shouldn’t have to. Every search deserves its own URL.


1 Thanks as always to Brewster Kahle for preserving what publishers did not.

Annotation-powered apps: A “Hello World” example

Workflows that can benefit from annotation exist in many domains of knowledge work. You might be a teacher who focuses a class discussion on a line in a poem. You might be a scientist who marks up research papers in order to reason about connections among them. You might be an investigative journalist fact-checking a claim. A general-purpose annotation tool like Hypothesis, already useful in these and other cases, can be made more useful when tailored to a specific workflow.

I’ve tried a few different approaches to that sort of customization. For the original incarnation of the Digital Polarization Project, I made a Chrome extension that streamlined the work of student fact-checkers. For example, it assigned tags to annotations in a controlled way, so that it was easy to gather all the annotations related to an investigation.

To achieve that effect the Digipo extension repurposed the same core machinery that the Hypothesis client uses to convert a selection in a web page into the text quote and text position selectors that annotations use to identify and anchor to selections in web pages. Because the Digipo extension and the Hypothesis extension shared this common machinery, annotations created by the former were fully interoperable with those created by the latter. The Hypothesis client didn’t see Digipo annotations as foreign in any way. You could create an annotation using the Digipo tool, with workflow-defined tags, and then open that annotation in Hypothesis and add a threaded discussion to it.

As mentioned in Annotations are an easy way to Show Your Work, I’ve been unbundling the ingredients of the Digipo tool to make them available for reuse. Today’s project aims to be the “Hello World” of custom annotation-powered apps. It’s a simple working example of a powerful three-step pattern:

  1. Given a selection in a page, form the selectors needed to post an annotation that targets the selection.
  2. Lead a user through an interaction that influences the content of that annotation.
  3. Post the annotation.

For the example scenario, the user is a researcher tasked with evaluating the emotional content of selected passages in texts. The researcher is required to label the selection as StrongPositive, WeakPositive, Neutral, WeakNegative, or StrongNegative, and to provide freeform comments on the tone of the selection and the rationale for the chosen label.

Here’s the demo in action:

The code behind the demo is here. It leverages two components. The first encapsulates the core machinery used to identify the selection. The second provides a set of functions that make it easy to create annotations and post them to Hypothesis. Some of these provide user interface elements, notably the dropdown list of Hypothesis groups that you can also see in other Hypothesis-oriented tools like facet and zotero. Others wrap the parts of the Hypothesis API used to search for, or create, annotations.

My goal is to show that you needn’t be a hotshot web developer to create a custom annotation-powered app. If the pattern embodied by this demo app is of interest to you, and if you’re at all familiar with basic web programming, you should be able to cook up your own implementations of the pattern pretty easily.

To work with selections in web pages, annotation-powered apps like these have to get themselves injected into web pages. There are three ways that can happen, and the Hypothesis client exploits all of them. Browser extensions are the most effective method. The Hypothesis extension is currently available only for Chrome, but there’s a Firefox extension waiting in the wings, made possible by an emerging consensus around the WebExtensions model.

A second approach entails sending a web page through a proxy that injects an annotation-powered app into the page. The Hypothesis proxy, via.hypothes.is, does that.

Finally there’s the venerable bookmarklet, a snippet of JavaScript that runs in the context of the page when the user clicks a button on the browser’s bookmarks bar. Recent evolution of the browser’s security model has complicated the use of bookmarklets. The simplest ones still work everywhere, but bookmarklets that load helper scripts — like the HelloWorldAnnotated demo — are thwarted by sites that enforce Content Security Policy.

It’s a complicated landscape full of vexing tradeoffs. Here’s how I recommend navigating it. The HelloWorldAnnotated demo uses the bookmarklet approach. If you’re an instructional designer targeting Project Gutenberg, or a scientific developer targeting the scientific literature, or a newsroom toolbuilder targeting most sources of public information, you can probably get away with deploying annotation-powered apps using bookmarklets. It’s the easiest way to build and deploy such apps. If the sites you need to target do enforce Content Security Policy, however, then you’ll need to deliver the same apps by way of a browser extension or a proxy. Of those two choices, I’d recommend a browser extension, and if there’s interest, I’ll do a follow-on post showing how to recreate the HelloWorldAnnotated demo in that form.

Annotating on, and with, media


At the 2018 I Annotate conference I gave a flash talk on the topic covered in more depth in Open web annotation of audio and video. These are my notes for the talk, along with the slides.


Here’s something we do easily and naturally on the web: Make a selection in a page.

The web annotation standard defines a few different ways to describe the selection. Here’s how Hypothesis does it:

We use a variety of selectors to help make the annotation target resilient to change. I’d like you to focus here on the TextPositionSelector, which defines the start and the end of the selection. Which is something we just take for granted in the textual domain. Of course a selection has a start and an end. How could it not?

An annotation like this gives you a kind of an URL that points to a selection in a document. There isn’t yet a standard way to write that URL along with the selectors, so Hypothesis points to that combination — that is, the URL plus the selectors — using an ID, like this:

The W3C have thought about a way to bundle selectors with the URL, so you’d have a standard way to cite a selection, like this:

In any case, the point is we’re moving into a world where selections in web resources have their own individual URLs.

Now let’s look again at this quote from Nancy Pelosi:

That’s not something she wrote. It’s something she said, at the Peter G Peterson Fiscal Summit, that was recorded on video.

Is there a transcript that could be annotated? I looked, and didn’t find anything better than this export of YouTube captions:

But of course we lack transcriptions for quite a lot of web audio and video. And lots of it can’t be transcribed, because it’s music, or silent moving pictures.

Once you’ve got a media selection there are some good ways to represent it with an URL. YouTube does it like this:

And with filetypes like mp3 and mp4, you can use media fragment syntax like this:

The harder problem in the media domain turns out to be just making the selection in the first place.

Here I am in that video, in the process of figuring out that the selection I’m looking for starts at 28:20 and ends at 28:36.

It’s not a fun or easy process. You can set the start, as I’ve done here, but then you have to scrub around on the timeline looking for the end, and then write that down, and then tack it onto the media fragment URL that you’ve captured.

It’s weird that something so fundamental should be missing, but there just isn’t an easy and natural way to make a selection in web media.

This is not a new problem. Fifteen years ago, these were the dominant media players.

We’ve made some progress since then. The crazy patchwork of plugins and apps has thankfully converged to a standard player that works the same way everywhere.

Which is great. But you still can’t make a selection!

So I wrote a tool that wraps a selection interface around the standard media player. It works with mp3s, mp4s, and YouTube videos. Unlike the standard player, which has a one-handled slider, this thing has a two-handled slider which is kind of obviously what you need to work with the start and end of a selection.

You can drag the handles to set the start and end, and you can nudge them forward and backward by minutes and seconds, and when you’re ready to review the intro and outro for your clip, you can play just a couple of seconds on both ends to check what you’re capturing.

When you’re done, you get a YouTube URL that will play the selection, start to stop, or an mp3 or mp4 media fragment URL that will do the same.

So how does this relate to annotation? In a couple of ways. You can annotate with media, or you can annotate on media.

Here’s what I mean by annotating with media.

I’ve selected some text that wants to be contextualized by the media quote, and I’ve annotated that text with a media fragment link. Hypothesis turns that link into an embedded player (thanks, by the way, to a code contribution from Steel Wagstaff, who’s here, I think). So the media quote will play, start to stop, in this annotation that’s anchored to a text selection at Politifact.

And here’s what I mean by annotating on media.

If I’m actually on a media URL, I just can annotate it. In this case there’s no selection to be made, the URL encapsulates the selection, so I can just annotate the complete URL.

This is a handy way to produce media fragment URLs that you can use in these ways. I hope someone will come up with a better one than I have. But the tool is begging to be made obsolete when the selection of media fragments becomes as easy and natural as the selection of text has always been.

Annotations are an easy way to Show Your Work

Journalists are increasingly being asked to show their work. Politifact does that with a sources box, like this:

This is great! The more citation of sources, the better. If I want to check those sources, though, I often wind up spending a lot of time searching within source articles to find passages cited implicitly but not explicitly. If those passages are marked using annotations, the method I’ll describe here makes that material available explicitly, in ways that streamline the reporter’s workflow and improve the reader’s experience.

In A Hypothesis-powered Toolkit for Fact Checkers I described a toolkit that supported the original incarnation of the Digital Polarization Project. More recently I’ve unbundled the key ingredients of that toolkit and made them separately available for reuse. The ingredient I’ll discuss here, HypothesisFootnotes, is illustrated in this short clip from a 10-minute screencast about the original toolkit. Here’s the upshot: Given a web page that contains Hypothesis direct links, you can include a script that pulls the cited material into the page, and connects direct links in the page to citations gathered elsewhere in the page.

Why would such links exist in the first place? Well, if you’re a reporter working on an investigation, you’re likely to be gathering evidence from a variety of sources on the web. And you’re probably copying and pasting bits and pieces of those sources into some kind of notebook. If you annotate passages within those sources, you’ve captured them in a way that streamlines your own personal (or team) management of that information. That’s the primary benefit. You’re creating a virtual notebook that makes that material available not only to you, but to your team. Secondarily, it’s available to readers of your work, and to anyone else who can make good use of the material you’ve gathered.

The example here brings an additional benefit, building on another demonstration in which I showed how to improve Wikipedia citations with direct links. There I showed that a Wikipedia citation need not only point to the top of a source web page. If the citation refers specifically to a passage within the source page, it can use a direct link to refer the reader directly to that passage.

Here’s an example that’s currently live in Wikipedia:

It looks like any other source link in Wikipedia. But here’s where it takes you in the cited Press Democrat story:

Not every source link warrants this treatment. When a citation refers to a specific context in a source, though, it’s really helpful to send the reader directly to that context. It can be time-consuming to follow a set of direct links to see cited passages in context. Why not collapse them into the article from which they are cited? That’s what HypothesisFootnotes does. The scattered contexts defined by a set of Hypothesis direct links are assembled into a package of footnotes within the article. Readers can still visit those contexts, of course, but since time is short and attention is scarce, it’s helpful to collapse them into an included summary.

This technique will work in any content management system. If you have a page that cites sources using direct links, you can use the HyothesisFootnotes script to bring the targets of those direct links into the page.

Investigating ClaimReview

In Annotating the Wild West of Information Flow I discussed a prototype of a ClaimReview-aware annotation client. ClaimReview is a standard way to format “a fact-checking review of claims made (or reported) in some creative work.” Now that both Google and Bing are ingesting ClaimReview markup, and fact-checking sites are feeding it to them, the workflow envisioned in that blog post has become more interesting. A fact checker ought to be able to highlight a reviewed claim in its original context, then capture that context and push it into the app that produces the ClaimReview markup embedded in fact-checking articles and slurped by search engines.

That workflow is one interesting use of annotation in the domain of fact-checking, it’s doable, and I plan to explore it. But here I’ll focus instead on using annotation to project claim reviews onto the statements they review, in original context. Why do that? Search engines may display fact-checking labels on search results, but nothing shows up when you land directly on an article that includes a reviewed claim. If the reviewed claim were annotated with the review, an annotation client could surface it.

To help make that idea concrete, I’ve been looking at ClaimReview markup in the wild. Not unexpectedly, it gets used in slightly different ways. I wrote a bookmarklet to help visualize the differences, and used it to find these five patterns:

https://www.washingtonpost.com/news/fact-checker/wp/2018/03/16/does-president-trumps-border-wall-pay-for-itself

{
  "urlOfFactCheck": "https://www.washingtonpost.com/news/fact-checker/wp/2018/03/16/does-president-trumps-border-wall-pay-for-itself/?utm_term=.31281569ab46",
  "reviewRating": {
    "@type": "Rating",
    "ratingValue": "-1",
    "alternateName": "Fuzzy math",
    "worstRating": "-1",
    "bestRating": "-1",
    "image": "https://dhpikd1t89arn.cloudfront.net/custom-rating/custom-ratings-b89013e8-729f-44aa-816e-93a9307b079c"
  },
  "itemReviewed": {
    "@type": "CreativeWork",
    "author": {
      "@type": "Person",
      "name": "Donald Trump ",
      "jobTitle": "President",
      "image": "https://dhpikd1t89arn.cloudfront.net/custom-rating/custom-ratings-b89013e8-729f-44aa-816e-93a9307b079c",
      "sameAs": [
        "www.whitehouse.gov"
      ]
    },
    "datePublished": "2018-03-13",
    "name": "in a tweet"
  },
  "claimReviewed": "\"The $18 billion wall will pay for itself by curbing the importation of crime, drugs and illegal immigrants who tend to go on the federal dole.”"
}

Did a Captured Islamic State Leader Have Barack Obama On Speed Dial?

{
  "urlOfFactCheck": "https://www.snopes.com/fact-check/did-captured-islamic-state-obama/",
  "reviewRating": {
    "alternateName": "False",
    "ratingValue": "",
    "bestRating": ""
  },
  "itemReviewed": {
    "datePublished": "2018-03-13T08:25:46+00:00",
    "name": "Snopes",
    "sameAs": "http://dailyworldupdate.com/2018/03/11/breaking-captured-isis-leader-had-obama-on-speed-dial/"
  },
  "claimReviewed": "A captured Islamic State leader's cell phone contained the phone numbers of world leaders, including former President Barack Obama."
}

Trump Misses with Mars Attack

{
  "urlOfFactCheck": "https://www.factcheck.org/2018/03/trump-misses-mars-attack/",
  "reviewRating": {
    "@type": "Rating",
    "ratingValue": "-1",
    "alternateName": "Clinton Endorsed Mars Flight",
    "worstRating": "-1",
    "bestRating": "-1",
    "image": "https://dhpikd1t89arn.cloudfront.net/custom-rating/custom-ratings-76945159-07c1-49d6-833e-717c1da2478f"
  },
  "itemReviewed": {
    "@type": "CreativeWork",
    "author": {
      "@type": "Person",
      "name": "Donald Trump",
      "jobTitle": "President of the United States",
      "image": "https://dhpikd1t89arn.cloudfront.net/custom-rating/custom-ratings-76945159-07c1-49d6-833e-717c1da2478f",
      "sameAs": [
        "http://www.whitehouse.gov"
      ]
    },
    "datePublished": "2018-03-13",
    "name": "Speech at military base"
  },
  "claimReviewed": "NASA “wouldn’t have been going to Mars if my opponent won.\""
}

http://www.politifact.com/texas/statements/2018/mar/15/ted-cruz/ted-cruz-says-beto-orourke-wants-open-borders-and-/

{
  "urlOfFactCheck": "http://www.politifact.com/texas/statements/2018/mar/15/ted-cruz/ted-cruz-says-beto-orourke-wants-open-borders-and-/",
  "reviewRating": {
    "@type": "Rating",
    "ratingValue": "2",
    "alternateName": "False",
    "worstRating": "1",
    "bestRating": "7",
    "image": "https://dhpikd1t89arn.cloudfront.net/rating_images/politifact/tom-false.jpg"
  },
  "itemReviewed": {
    "@type": "CreativeWork",
    "author": {
      "@type": "Person",
      "name": "Ted Cruz",
      "jobTitle": "U.S. senator for Texas",
      "image": "https://dhpikd1t89arn.cloudfront.net/rating_images/politifatc/tom-false.jpg",
      "sameAs": [
        "https://www.youtube.com/watch?v=MvKElqdUxZg"
      ]
    },
    "datePublished": "2018-03-06",
    "name": "Houston, Texas"
  },
  "claimReviewed": "Says Beto O’Rourke wants \"open borders and wants to take our guns.\""
}

Breitbart repeats blogger’s unsupported claim that NOAA manipulates data to exaggerate warming

{
  "urlOfFactCheck": "https://climatefeedback.org/claimreview/breitbart-repeats-bloggers-unsupported-claim-noaa-manipulates-data-exaggerate-warming/",
  "reviewRating": {
    "@type": "Rating",
    "alternateName": "Unsupported",
    "bestRating": 5,
    "ratingValue": 2,
    "worstRating": 1
  },
  "itemReviewed": {
    "@type": "CreativeWork",
    "author": {
      "@type": "Person",
      "name": "James Delingpole"
    },
    "headline": "TBD",
    "image": {
      "@type": "ImageObject",
      "height": "200px",
      "url": "https://climatefeedback.org/wp-content/uploads/Sources/Delingpole_Breitbart_News.png",
      "width": "200px"
    },
    "publisher": {
      "@type": "Organization",
      "logo": "Breitbart",
      "name": "Breitbart"
    },
    "datePublished": "20 February 2018",
    "url": "TBD"
  },
  "claimReviewed": "NOAA has adjusted past temperatures to look colder than they were and recent temperatures to look warmer than they were."
}

To normalize the difference I’ll need to look at more examples from these sites. But I’m also looking for more patterns, so if you know of other sites that routinely embed ClaimReview markup, please drop me links!

 

Open web annotation of audio and video

Hypothesis does open web annotation of text. Let’s unpack those concepts one at a time.

Open, a famously overloaded word, here means:

  • The software that reads, writes, stores, indexes, and searches for annotations is available under an open source license.

  • Open standards govern the representation and exchange of annotations.

  • The annotation system is a good citizen of the open web.

Web annotation, in the most general sense, means:

  • Selecting something within a web resource.

  • Attaching stuff to that selection.

Text, as the Hypothesis annotation client understands it, is HTML, or PDF transformed to HTML. In either case, it’s what you read in a browser, and what you select when you make an annotation.

What’s the equivalent for audio and video? It’s complicated because although browsers enable us to select passages of text, the standard media players built into browsers don’t enable us to select segments of audio and video.

It’s trivial to isolate a quote in a written document. Click to set your cursor to the beginning, then sweep to the end. Now annotation can happen. The browser fires a selection event; the annotation client springs into action; the user attaches stuff to the selection; the annotation server saves that stuff; the annotation client later recalls it and anchors it to the selection.

But selection in audio and video isn’t like selection in text. Nor is it like selection in images, which we easily and naturally crop. Selection of audio and video happens in the temporal domain. If you’ve ever edited audio or video you’ll appreciate what that means. Setting a cursor and sweeping a selection isn’t enough. You can’t know that you got the right intro and outro by looking at the selection. You have to play the selection to make sure it captures what you intended. And since it probably isn’t exactly right, you’ll need to make adjustments that you’ll then want to check, ideally without replaying the whole clip.

All audio and video editors support making and checking selections. But what’s built into modern browsers is a player, not an editor. It provides a slider with one handle. You can drag the handle back and forth to play at different times, but you can’t mark a beginning and an end.

YouTube shares that limitation, by the way. It’s great that you can right-click and Copy video URL at current time. But you can’t mark that as the beginning of a selection, and you can’t mark a corresponding end.

We mostly take this limitation for granted, but as more of our public discourse happens in audio or video, rarely supported by written transcripts, we will increasingly need to be able to cite quotes in the temporal domain.

Open web annotation for audio and video will, therefore, require standard players that support selection in the temporal domain. I expect we’ll get there, but meanwhile we need to provide a way to do temporal selection.

In Annotating Web Audio I described a first cut at a clipping tool that wraps a selection interface around the standard web audio and video players. It was a step in the right direction, but was too complex — and too modal — for convenient use. So I went back to the drawing board and came up with a different approach shown here:

This selection tool has nothing intrinsically to do with annotation. It’s job is to make your job easier when you are constructing a link to an audio or video segment.

For example, in Welcome to the Sapiezoic I reflected on a Long Now talk by David Grinspoon. Suppose I want to support that post with an audio pull quote. The three-and-a-half-minute segment I want to use begins: “The Anthropocene has been proposed as a new epoch…” It ends: “…there has never been a geological force aware of its own existence. And to me that’s a very profound change.”

Try opening http://podcast.longnow.org/salt/salt-020170906-grinspoon-podcast.mp3 and finding the timecodes for the beginning (“The Anthropocene…”) and the end (“…profound change”). The link you want to construct is: http://podcast.longnow.org/salt/salt-020170906-grinspoon-podcast.mp3#t=730,949. That’s a Media Fragments link. It tells a standard media player when to start and stop.

If you click the bare link, your browser will load the audio file into its standard player and play the clip. If you paste the link into a piece of published text, it may even be converted by the publishing system into an inline player. Here’s an annotation on my blog post that does that:

That’s a potent annotation! But to compose it you have to find where the quote starts (12:10) and ends (15:49), then convert those timecodes to seconds. Did you try? It’s doable, but so fiddly that you won’t do it easily or routinely.

The tools at http://jonudell.net/av/ aim to lower that activation threshold. They provide a common interface for mp3 audio, mp4 video, and YouTube video. The interface embeds a standard audio or video player, and adds a two-handled slider that marks the start and end of a clip. You can adjust the start and end and hear (or hear and see) the current intro and outro. At every point you’ve got a pair of synced permalinks. One is the player link, a media fragment URL like http://podcast.longnow.org/salt/salt-020170906-grinspoon-podcast.mp3#t=730,949. The other is the editor link. It records the settings that produce the player link.

As I noted in Annotating Web Audio, these clipping tools are just one way to ease the pain of constructing media fragment URLs. Until standard media players enable selection, we’ll need complementary tools like these to help us do it.

Once we have a way to construct timecoded segments, we can return to the question: “What is open web annotation for audio and video?”

At the moment, I see two complementary flavors. Here’s one:

I’m using a Hypothesis page note to annotate a media fragment URL. There’s no text to which Hypothesis can anchor an annotation, but it can record a note that refers to the whole page. In effect I’m bookmarking the page as I could also do in, for example, Pinboard. The URL encapsulates the selection; annotations that attach to the URL refer to the selection.

This doesn’t yet work as you might expect in Hypothesis. If you visit http://podcast.longnow.org/salt/salt-020170906-grinspoon-podcast.mp3#t=730,949 with Hypothesis you’ll see my annotation, but you’ll also see it if you visit the same URL at #t=1,10, or any other media fragment. That’s helpful in one way, because if there were multiple annotations on the podcast you’d want to discover all of them. But it’s unhelpful in a more important way. If you want to annotate a particular segment for personal use, or to mark it as a pull quote, or because you’re a teacher leading a class discussion that refers to that segment, then you want annotations to refer to the particular segment.

Why doesn’t Hypothesis distinguish between #t=1,10 and #t=730,949? Because what follows the # in a URL is most often not a media fragment, it’s an ordinary fragment that marks a location in text. You’ve probably seen how that works. A page like example.com/page1 has intrapage links like example.com/page1#section1 and example.com/page1#section2. I can send you to those locations in the page by capturing and relaying those fragment-enhanced links. In that case, if we’re annotating the page, we’d most likely want all our annotations to show up, no matter which fragment is focused in the browser. So Hypothesis doesn’t record the fragment when it records the target URL for an annotation.

But there can be special fragments. If I share a Hypothesis link, for example https://hyp.is/uT06DvGBEeeXDlMfjMFDAg, you’ll land on a link that ends with #annotations:uT06DvGBEeeXDlMfjMFDAg. The Hypothesis client uses that information to scroll the annotated document to the place where the annotation is anchored.

Media fragments could likewise be special. The Hypothesis server, which normally discards media fragments, could record them so that annotations made on a media fragment URL would target the fragment. And I think that’s worth doing. Meanwhile, note that you can annotate the editor links provided by the tools at http://jonudell.net/av/:

This works because the editor links don’t use fragments, only URL parameters, and because Hypothesis regards each uniquely-parameterized URL as a distinct annotation target. Note also that you needn’t use the tools as hosted on my site. They’re just a small set of files that can be hosted anywhere.

Eventually I hope we’ll get open web annotation of audio and video that’s fully native, meaning not only that the standard players support selection, but also that they directly enable creation and viewing of annotations. Until then, though, this flavor of audio/video annotation — let’s call it annotating on media — will require separate tooling both for selecting quotes, and for creating and viewing annotations directly on those quotes.

We’ve already seen the other flavor: annotating with media. To do that with Hypothesis, construct a media fragment URL and cite it in a Hypothesis annotation. What should the annotation point to? That’s up to you. I attached the David Grinspoon pull quote to one of my own blog posts. When I watched a PBS interview with Virginia Eubanks, I captured one memorable segment and attached it to the page on her blog that features the book discussed in the interview.

(If I were Virginia Eubanks I might want to capture the pull quote myself, and display it on my book page for visitors who aren’t seeing it through the Hypothesis lens.)

Open web annotation of audio and video should encompass both of these flavors. You should be able to select a clip within a standard player, and annotate it in situ. And you should be able to use that clip in an annotation.

Until players enable selection, the first flavor — annotating on a segment — will require a separate tool. I’ve provided one implementation, there can be (perhaps already are?) others. However it’s captured, the selection will be represented as a media fragment link. Hypothesis doesn’t yet, but pretty easily could, support annotation of such links in a way that targets media fragments.

The second flavor — annotation with a segment — again requires a way to construct a media fragment link. With that in hand, you can just paste the link into the Hypothesis annotation editor. Links ending with .mp3#t=10,20 and links like youtube.com/watch?v=Avxm7JYjk8M&start=10&end=20 will become embedded players that start and end at the indicated times. Links like .mp4#t=10,20 and youtube.com/embed/Avxm7JYjk8M?start=10&end=20 don’t yet become embedded players but can.

The ideal implementation of open web annotation for audio and video will have to wait for a next generation of standard media players. But you can use Hypothesis today to annotate on as well as with media.

How to improve Wikipedia citations with Hypothesis direct links

Wikipedia aims to be verifiable. Every statement of fact should be supported by a reliable source that the reader can check. Citations in Wikipedia typically refer to online documents accessible at URLs. But with the advent of standard web annotation we can do better. We can add citations to Wikipedia that refer precisely to statements that support Wikipedia articles.

According to Wikipedia’s policy on citing sources:

Wikipedia’s Verifiability policy requires inline citations for any material challenged or likely to be challenged, and for all quotations, anywhere in article space.

Last night, reading https://en.wikipedia.org/wiki/Tubbs_Fire, I noticed this unsourced quote:

Sonoma County has four “historic wildfire corridors,” including the Hanly Fire area.

I searched for the source of that quotation, found it in a Press Democrat story, annotated the quote, and captured a Hypothesis direct link to the annotation. In this screenshot, I’ve clicked the annotation’s share icon, and then clicked the clipboard icon to copy the direct link to the clipboard. The direct link encapsulates the URL of the story, plus the information needed to locate the quotation within the story.

Given such a direct link, it’s straightforward to use it in a Wikipedia citation. Back in the Wikipedia page I clicked the Edit link, switched to the visual editor, set my cursor at the end of the unsourced quote, and clicked the visual editor’s Cite button to invoke this panel:

There I selected the news template, and filled in the form in the usual way, providing the title of the news story, its date, its author, the name of the publication, and the date on which I accessed the story. There was just one crucial difference. Instead of using the Press Democrat URL, I used the Hypothesis direct link.

And voilà! There’s my citation, number 69, nestled among all the others.

Citation, as we’ve known it, begs to be reinvented in the era of standard web annotation. When I point you to a document in support of a claim, I’m often thinking of a particular statement in that document. But the burden is on you to find that statement in the document to which my citation links. And when you do, you may not be certain you’ve found the statement implied by my link. When I use a direct link, I relieve you of that burden and uncertainty. You land in the cited document at the right place, with the supporting statement highlighted. And if it’s helpful we can discuss the supporting statement in that context.

I can envision all sorts of ways to turbocharge Wikipedia’s workflow with annotation-powered tools. But no extra tooling is required to use Hypothesis and Wikipedia in the way I’ve shown here. If you find an unsourced quote in Wikipedia, just annotate it in its source context, capture the direct link, and use it in the regular citation workflow. For a reader who clicks through Wikipedia citations to check original sources, this method yields a nice improvement over the status quo.

Annotating Web Audio

On a recent walk I listened to Unmasking Misogyny on Radio Open Source. One of the guests, Danielle McGuire, told the story of Rosa Parks’ activism in a way I hadn’t heard before. I wanted to capture that segment of the show, save a link to it, and bookmark the link for future reference.

If you visit the show page and click the download link, you’ll load the show’s MP3 file into your browser’s audio player. Nowadays that’s almost always going to be the basic HTML5 player. Here’s what it looks like in various browsers:

The show is about an hour long. I scrubbed along the timeline until I heard Danielle McGuire’s voice, and then zeroed in on the start and end of the segment I wanted to capture. It starts at 18:14 and ends at 21:11. Now, how to link to that segment?

I first investigated this problem in 2004. Back then, I learned that it’s possible to fetch and play random parts of MP3 files, and I made a web app that would figure out, given start and stop times like 18:14 and 21:11, which part of the MP3 file to fetch and play. Audio players weren’t (and still aren’t) optimized for capturing segments as pairs of minute:second parameters. But once you acquired those parameters, you could form a link that would invoke the web app and play the segment. Such links could then be curated, something I often did using the del.icio.us bookmarking app.

Revisiting those bookmarks now, I’m reminded that Doug Kaye and I were traveling the same path. At ITConversations he had implemented a clipping service that produced URLs like this:

http://www.itconversations.com/clip.php?showid=378&start=4:37&stop=6:04

Mine looked like this:

http://udell.infoworld.com/?url=http://www.itconversations.com/audio/download/ITConversations-378.mp3&beg=4:37&end=6:04

Both of those audio-clipping services are long gone. But the audio files survive, thanks to the exemplary cooperation between ITConversations and the Internet Archive. So now I can resurrect that ITConversations clip — in which Doug Engelbart, at the Accelerating Change conference in 2004, describes the epiphany that inspired his lifelong quest — like so:

http://jonudell.net/av/audio.html?url=http://jonudell.net/audio/ITC.AC2004-DougEngelbart-2004.11.07.mp3&startmin=4&startsec=37&endmin=6&endsec=4

And here’s the segment of Danielle McGuire’s discussion of Rosa Parks that I wanted to remember:

http://jonudell.net/av/audio.html?url=http://ia601506.us.archive.org/5/items/171207OSPODCASTSexualHarassment/171207-OS-PODCAST-SexualHarassment.mp3&startmin=18&startsec=14&endmin=21&endsec=11

This single-page JavaScript app aims to function both as a player of predefined segments, and as a tool that makes it as easy as possible to define segments. It’s still a work in progress, but I’m able to use it effectively even as I continue to refine the interaction details.

For curation of these clips I am, of course, using Hypothesis. Here are some of the clips I’ve collected on the tag AnnotatingAV:

To create these annotations I’m using Hypothesis page notes. An annotation of this type is like a del.icio.us or pinboard.in bookmark. It refers to the whole resource addressed by a URL, rather than to a segment of interest within a resource.

Most often, a Hypothesis user defines a segment of interest by selecting a passage of text in a web document. But if you’re not annotating any particular selection, you can use a page note to comment on, tag, and discuss the whole document.

Since each audio clip defines a segment as a standalone web page with a unique URL, you can use a Hypothesis page note to annotate that standalone page:

It’s a beautiful example of small pieces loosely joined. My clipping tool is just one way to form URLs that point to audio and video segments. I hope others will improve on it. But any clipping tool that produces unique URLs can work with Hypothesis and, of course, with any other annotation or curation tool that targets URLs.

Syndicating annotations

Steel Wagstaff asks:

Immediate issue: we’ve got books on our dev server w/ annotations & want to move them intact to our production instance. The broader use case: I publish an open Pressbook & users make public comments on it. Someone else wants to clone the book including comments. How?

There are currently three URL-independent identifiers that can be used to coalesce annotations across instances of a web document published at different URLs. The first was the PDF fingerprint, the second was the DOI, and a third, introduced recently as part of Hypothesis’ EPUB support, uses Dublin Core metadata like so:

<meta name=”dc.identifier” content=”xchapter_001″>
<meta name=”dc.relation.ispartof” content=”org.example.hypothesis.demo.epub-samples.moby-dick-basic”>

If you dig into our EPUB.js and Readium examples, you’ll find those declarations are common to both instances of chapter 1 of Moby Dick. Here’s an annotation anchored to the opening line, Call me Ishmael. When the Hypothesis client loads, in a page served from either of the example URLs, it queries for two identifiers. One is the URL specific to each instance. The other is a URN formed from the common metadata, and it looks like this:

urn:x-dc:org.example.hypothesis.demo.epub-samples.moby-dick-basic/chapter_001

When you annotate either copy, you associate its URL with this Uniform Resource Name (URN). You can search for annotations using either of the URLs, or the just URN like so:

https://hypothes.is/search?q=url:urn:x-dc:org.example.hypothesis.demo.epub-samples.moby-dick-basic/xchapter_001

Although it sprang to life to support ebooks, I think this mechanism will prove more broadly useful. Unlike PDF fingerprints and DOIs, which typically identify whole works, it can be used to name chapters and sections. At a conference last year we spoke with OER (open educational resource) publishers, including Pressbooks, about ways to coalesce annotations across their platforms. I’m not sure this approach is the final solution, but it’s usable now, and I hope pioneers like Steel Wagstaff will try it out and help us think through the implications.

Really, AT&T?

We woke up this morning in Santa Rosa to smoke and sirens. Last night’s winds fanned a bunch of wildfires in the North Bay, and parts of our town are destroyed. We’re a few miles south of the evacuation zone, things might shift around, but at the moment we’re staying put.

Information was hard to come by. Our Comcast service is down. Our AT&T phones were up and running though, so I turned on my mobile hotspot and read this: “Cannot turn on hotspot, please visit att.com or call 611.”

WTF?

Here’s what happened. Last month we started hitting data overage charges. That hadn’t been an issue before, but we dropped by the AT&T store to review our options. The sales rep pushed hard for an upgrade to an unlimited data plan for an extra $5/mo. Not really necessary, we’re almost always on WiFi, but OK, sure, five bucks, why not?

Turns out the rep neglected to mention that the upgrade removed our tethering capability. This is not something you want to find out while breathing smoke, hearing sirens, and trying to make sense of the latest evacuation map for your burning city. According to the rep we spoke with today, this critical fact about tethering is often omitted from the upsell pitch.

We’ve got it turned back on now. I’m awaiting a callback from a manager’s manager about the additional $30/mo they plan to charge to restore a capability they hadn’t told me they were taking away.

This is a small thing. We are, of course, infinitely luckier than a bunch of folks in our town who will return to the charred foundations of what were their homes. We’re mainly thinking about them today. But while we’re waiting for the ash to settle, I just want to say: Really, AT&T?

Welcome to the Sapiezoic

The latest Long Now podcast, by David Grinspoon, takes a very long view indeed. As we transition from the Holocene to the Anthropocene, he thinks, we’re not just entering a new geological epoch, as shown here:

That alone would be a big deal. But epochs are just geologic eyeblinks a few million years in duration. Grinspoon thinks we might be entering something way bigger. Not just a period or an era. We might happen to be alive now at an eon boundary, as shown here:

There have only been four eons so far. Each was a major transition in earth history — a shift in the relationship between life and the planet. Life first emerged at the beginning of the Archean era around 4 billion years ago, when things cooled down enough. Around 2.5 billion years ago, cyanobacteria learned to photosynthesize. They bathed the world in oxygen, caused mass extinction, and deeply entangled life with the physical and chemical workings of the planet. At the boundary between the Proterozoic and Phanerozoic, around 500 million years ago, life went multicellular, plants and animals appeared, and the modern eon began.

Will our descendants look back on the Anthropocene as the dawn of a fifth eon? Grinspoon makes a compelling case that they might. The Archean cyanobacteria that poisoned the environment couldn’t know what they were doing. We can. Our infrastructure is taking over the planet as surely as the oxygenated atmosphere did. But it is, at least potentially, under our conscious control. The hallmark of the Sapiezoic eon he envisions: intentional re-terraforming.

Grinspoon cites one shining example: the Montreal Protocol that will, if we stay the course, reverse manmade ozone depletion. “We are as gods,” says Stewart Brand, “so we might as well get good at it.”

I’ve listened to all the Long Now podcasts, some more than once. This one rates very highly. It’s a great talk.

Fact-checking Naomi Klein’s “No Is Not Enough”

On a hike last week I heard an excellent episode of Radio Open Source, featuring Naomi Klein, David Graeber, and Pankaj Mishra. One of the segments of interest that stuck with me is this remark by Naomi Klein:

We need to examine the way in which politics has been taken over by the logic of corporate branding, which is not something Trump started. Trump was just better at it than anybody else because he is himself a fully commercialized brand. So the table was set for Trump, he just showed up and said, “Well, I know this game better than you jokers, I’m the real thing, I’m a reality TV star and I’m a megabrand. Step aside!”

(If I could, I would link you directly to that segment in situ, that’s something I had working a long time ago, but since audio quotation still isn’t a ubiquitous feature of the web, here’s the compelling minute of audio that contains that quote.)

I was previously aware of Naomi Klein but had never heard her speak, had read none of her books, and was only slightly familiar with her critique of corporatized politics. Her conversation with Chris Lydon on that podcast prompted me to read her new book, No Is Not Enough, published just a few weeks ago.

I was also slightly familiar with criticism of Klein’s views. So, in a moment when the president of the United States had just tweeted a video of himself performing a mock attack during his time as a reality TV personality on the pro wrestling circuit, I was curious to know her thoughts but also prepared to take them with a grain of salt.

Here’s the book’s table of contents:

The first section elaborates on the above quote. Human megabrands are, Klein points out, a relatively new thing. She writes:

People keep asking — is he going to divest? Is he going to sell his businesses? Is Ivanka going to? But it’s not at all clear what these questions even mean, because their primary businesses are their names. You can’t disentangle Trump the man from Trump the brand; those two entities merged long ago. Every time he sets foot in one of his properties — a golf club, a hotel, a beach club — White House press corps in tow, he is increasing his overall brand value, which allows his company to sell more memberships, rent more rooms, and increase fees.

I hope we can agree across ideologies that this kind of thing is unhealthy. In the audio clip I cited above, Klein notes that the antidote is not a liberal megabrand, not Zuckerberg or Oprah. Conflation of brand power and political power is just a bad idea, and we need to reckon with that.

The rest of the book builds on arguments made in her earlier ones: Capitalism’s winners exploit natural and man-made crises to consolidate their winnings (The Shock Doctrine); climate change presents an existential challenge to that world order (This Changes Everything). Since I haven’t read those books, and have only just now read a few reviews pro and con, I lack the full context needed to evaluate the arguments in No Is Not Enough. But that’s exactly the right setup for the point about fact-checking that I want to make here.

How reliable is Naomi Klein on facts? I came to No Is Not Enough with no strong opinion one way or another. I raised an eyebrow, though, when I read this passage about Treasury secretary Steven Mnuchin:

Even among Goldman alumni, Steven Mnuchin has distinguished himself by his willingness to profit off misery. Afer the 2008 Wall Street collapse, and in the middle of the foreclosure crisis, Mnuchin purchased a California bank. The renamed company, OneWest, earned Mnuchin the nickname “Foreclosure King,” reportedly collecting $1.2 billion from the government to help cover the losses for foreclosed homes and evicting tens of thousands of people between 2009 and 2014. One attempted foreclosure involved a ninety-year-old woman who was behind on her payments by 27 cents.”

The last sentence sent me to Google, where I quickly learned it had been debunked in a tweetstorm by Ted Frank in January 2017. He works for a libertarian think tank, and I doubt we’d see eye to eye on many issues, but his takedown of the 27-cent claim was accurate. Politico, for example, corrected its version of the story.

This is unfortunate because everything else in the above quote seems to check out. And you don’t have to be a liberal snowflake to worry legitimately about the Goldmanization of the US Cabinet.1

I went on to spot-check a number of other claims in No Is Not Enough and again, so far as I can tell with modest effort, everything checks out. So my conclusion is that Klein, who says she wrote this book quickly, to respond to the current moment, with less attention to endnotes than usual, is generally reliable on facts.

The way in which I reached that conclusion is a pretty good example of the strategies outlined in Web Literacy for Student Fact-Checkers, and a reminder that those methods aren’t just for students. All of us — me, you, Naomi Klein, everyone — need to build those muscles and exercise them regularly.


1. On another episode of Radio Open Source, in a remarkable dialogue between Pat Buchanan and Ralph Nader, the arch-conservative Buchanan aligned with the arch-liberal Nader on that point:

I agree completely with Ralph, I did not know we were going to make the world safe for Goldman Sachs, and I am a little surprised to find three or four or five of these guys, one or two might have been OK.

Thoughts on Audrey Watters’ “Thoughts on Annotation”

Back in April, Audrey Watters’ decided to block annotation on her website. I understand why. When we project our identities online, our personal sites become extensions of our homes. To some online writers, annotation overlays can feel like graffiti. How can we respect their wishes while enabling conversations about their writing, particularly conversations that are intimately connected to the writing? At the New Media Consortium conference recently, I was finally able to meet Audrey in person, and we talked about how to balance these interests. Yesterday Audrey posted her thoughts about that conversation, and clarified a key point:

You can still annotate my work. Just not on my websites.

Exactly! To continue that conversation, I have annotated that post here, and transcluded my initial set of annotations below.


judell 6/27/2017 #

using an HTML meta tag to identify annotation preferences

This is just a back-of-the-napkin sketch of an idea, not a formal proposal.

judell 6/27/2017 #

I’m much less committed to having one canonical “place” for annotations than Hypothesis is

Hypothesis isn’t committed to that either. The whole point of the newly-minted web annotation standard is to enable an ecosystem of interoperable annotation clients and servers, analogous to comparable ecosystems of email and web clients and servers.

judell 6/27/2017 #

Hypothesis annotations of a PDF can be centralized, no matter where the article is hosted or whether it’s a local copy

Centralization and decentralization are slippery terms. I would rather say that Hypothesis can unify a set of annotations across a family of representations of the “same” work. Some members of that family might be HTML pages, others might be PDFs hosted on the web or kept locally.

It’s true that when Hypothesis is used to create and view such annotations, they are “centralized” in the Hypothesis service. But if someone else stands up an instance of Hypothesis, that becomes a separate pool of annotations. Likewise, we at Hypothesis have planned for, and expect to see, a world in which non-Hypothesis-based implementations of standard annotation capability will host still other separate pools of annotations.

So you might issue three different API queries — to Hypothesis, to a Hypothesis-based service, and to a non-Hypothesis-based service — for a PDF fingerprint or a DOI. Each of those services might or might not internally unify annotations across a family of “same” resources. If you were to then merge the results of those three queries, you’d be an annotation aggregator — the moral equivalent of what Radio UserLand, Technorati, and other blog aggregators did in the early blogosphere.

Dumb servers for personal clouds

I’m delighted to hear that my daughter and her best friend will be collaborating on a blog. And of course I’m tickled that she asked my advice on where to run it. I noted that Ghost is the new kid on the block, and is much simpler than what WordPress has become. But they want to do it for free, so WordPress it is.

Then she surprised me with this narrative:

I heard it’s better if you self-host, so that’s what we’ll want to do, right? I think self-hosting is good because you don’t have the website name in your blog URL. Also, more importantly, I think it’s how you ensure that it’s actually yours.

It turns out that she’d conflated self-hosting, i.e. running your own instance of the WordPress software and database, with the simpler method my own blog exemplifies. I use WordPress.com precisely because, although I do run my own servers, the fewer the better. I’m happy to rely on WordPress to host my blog for me. I’m also happy to pay them $13/year to connect jonudell.wordpress.com to blog.jonudell.net.

So that’ll be the solution for my daughter. But I’m left wondering how many others conflate self-hosting with domain redirection, and how that affects their thinking about control of their own digital identities and data. I suspect it’s often unclear that, whether you run a blog on WordPress.com or on another provider’s server, your data is equally under your control. Likewise, use of a personal domain name is equally possible in both cases. What is the difference? With self-hosting, you can use arbitrary WordPress plugins and themes, and/or modify the software. Sometimes, for some people, that matters. Often, for many, it doesn’t.

That said, I agree with Mike Caulfield’s plea to make servers dumb again. In my ideal world, I’d not only outsource the management of the blog software to WordPress, but would also connect the software to my personal cloud, which would be implemented by my chosen storage provider.

I got this idea from Gorden Bell’s MyLifeBits, and riffed on it to imagine cloud-hosted lifebits. Jim Groom ably summed up the argument here:

Will we ever get there? It has to happen sooner or later. Maybe, as Doug Levin suggests today, it’ll be sooner.

Celebrating Infrastructure

When cycling in forested New England countryside I sometimes wondered about the man-made forest built along the roadside — telephone poles, power lines, transformers — and thought someone should write a book about the industrial landscape. It turns out that someone did. Brian Hayes spent many years traveling around America, researching and photographing the infrastructure that sustains our civilization. The book he produced, Infrastructure: A Field Guide to the Industrial Landscape (2005, 2nd ed. 2014), is everything I imagined it would be.

(I found the book by way of a comment that Brian Hayes left here on this blog. “Couldn’t be that Brian Hayes,” I thought. But his signature led me to his blog and thence to Infrastructure‘s home on the web. I’m passing it along here in part to remind myself that my favorite books often aren’t new or well publicized. I find them serendipitously after they’ve been around for a while.)

My father and his twin brother were students of nature in a way I’ve never been. Their knowledge of plants and animals was encyclopedic and ever-expanding. But for most of us, the natural landscape is not an expanse of unnamed and unknown objects. We recognize egrets, crows, hummingbirds, oaks, pines, and maples. The same isn’t true of the industrial landscape. More often than not, driving along some industrial corridor, we’re likely to ask the question Brian Hayes’ daughter asked him: “What’s that thing?” Infrastructure answers those questions for her, and for us.

Chapters on mining, waterworks, farming, energy production and distribution, transportation, shipping, and waste management follow a plan that “traces the flow of materials, information, and energy” throughout the web of industrial networks. We learn how industrial processes work, and how to identify the structures that house and implement them. Not all of us encounter quarries, mills, dams, refineries, or power plants on a daily basis. But water towers, roads, bridges, power lines, and data cables are as much a part of our landscape as what nature put there.

Hayes invites us to know more about the names, appearances, and workings of the industrial landscape. He also challenges us to reconsider how we feel about that landscape.

I stood by the side of a highway near Gallup, New Mexico, looking on a classic vista of the American West: red sandstone buttes, rising from a valley floor. … In front of the cliffs, and towering over them, were several cylindrical spires that I recognized as petroleum fractionating columns; off to one side was a grove of gleaming white spherical tanks. … I suspect that most viewers of this scene would consider the industrial hardware to be an intrusion, a distraction, perhaps even a desecration of the landscape.

Guilty as charged. But I’m provoked by this book to reconsider. Celebrate infrastructure, don’t hide it, Stewart Brand tweeted today. “It is civilization’s metabolism and should be its pride..

Weaving the annotated web

In 1997, at the first Perl Conference, which became OSCON the following year, my friend Andrew Schulman and I both gave talks on how the web was becoming a platform not only for publishing, but also for networked software.

Here’s the slide I remember from Andrew’s talk:

http://wwwapps.ups.com/tracking/tracking.cgi?tracknum=1Z742E220310270799

The only thing on it was a UPS tracking URL. Andrew asked us to stare at it for a while and think about what it really meant. “This is amazing!” he kept saying, over and over. “Every UPS package now has its own home page on the world wide web!”

It wasn’t just that the package had a globally unique identifier. It named a particular instance of a business process. It made the context surrounding the movement of that package through the UPS system available to UPS employees and customers who accessed it in their browsers. And it made that same context available to the Perl programs that some of us were writing to scrape web pages, extract their data, and repurpose it.

As we all soon learned, URLs can point to many kinds of resources: documents, interactive forms, audio or video.

The set of URL-addressable resources has two key properties: it’s infinite, and it’s interconnected. Twenty years later we’re still figuring out all the things you can do on a web of hyperlinked resources that are accessible at well-known global addresses and manipulated by a few simple commands like GET, POST, and DELETE.

When you’re working in an infinitely large universe it can seem ungrateful to complain that it’s too small. But there’s an even larger universe of resources populated by segments of audio and video, regions of images, and most importantly, for many of us, text in web documents: paragraphs, sentences, words, table cells.

So let’s stare in amazement at another interesting URL:

https://hyp.is/LoaMFCSJEee3aAMJuXhO-w/www.ics.uci.edu/~fielding/pubs/dissertation/software_arch.htm

Here’s what it looks like to a human who follows the link: You land on a web page, in this case Roy Fielding’s dissertation on web architecture, it scrolls to the place where I’ve highlighted a phrase, and the Hypothesis sidebar displays my annotation which includes a comment and a tag.

And here’s what that resource looks like to a computer when it fetches a variant of that link:

{
    "body": [
        {
            "type": "TextualBody",
            "value": "components: web resources\n\nconnectors: links\n\ndata: data",
            "format": "text/markdown"
        },
        {
            "type": "TextualBody",
            "purpose": "tagging",
            "value": "IAnnotate2017"
        }
    ],
    "target": [
        {
            "source": "https://www.ics.uci.edu/~fielding/pubs/dissertation/software_arch.htm",
            "selector": [
                {
                    "type": "XPathSelector",
                    "value": "/table[2]/tbody[1]/tr[1]/td[1]",
                    "refinedBy": {
                        "start": 82,
                        "end": 114,
                        "type": "TextPositionSelector"
                    }
                },
                {
                    "type": "TextPositionSelector",
                    "end": 4055,
                    "start": 4023
                },
                {
                    "exact": "components, connectors, and data",
                    "prefix": "tion of architectural elements--",
                    "type": "TextQuoteSelector",
                    "suffix": "--constrained in their relations"
                }
            ]
        }
    ],
    "created": "2017-04-18T22:48:46.756821+00:00",
    "@context": "http://www.w3.org/ns/anno.jsonld",
    "creator": "acct:judell@hypothes.is",
    "type": "Annotation",
    "id": "https://hypothes.is/a/LoaMFCSJEee3aAMJuXhO-w",
    "modified": "2017-04-18T23:03:54.502857+00:00"
}

The URL, which we call a direct link, isn’t itself a standard way to address a selection of text, it’s just a link that points to a web resource. But the resource it points to, which describes the highlighted text and its coordinates within the document, is — since February of this year — a W3C standard. The way I like to think about it is that the highlighted phrase — and every possible highlighted phrase — has its own home page on the web, a place where humans and machines can jointly focus attention.

If we think of the web we’ve known as a kind of fabric woven together with links, the annotated web increases the thread count of that fabric. When we weave with pieces of URL-addressable documents, we can have conversations about those pieces, we can retrieve them, we can tag them, and we can interconnect them.

Working with our panelists and others, it’s been my privilege to build a series of annotation-powered apps that begin to show what’s possible when every piece of the web is addressable in this way.

I’ll show you some examples, then invite my collaborators — Beth Ruedi from AAAS, Mike Caulfield from Washington State University Vancouver, Anita Bandrowski from SciCrunch, and Maryann Martone from UCSD and Hypothesis — to talk about what these apps are doing for them now, and where we hope to take them next.

Science in the Classroom

First up is a AAAS project called Science in the Classroom, a collection of research papers from the Science family of journals that are annotated — by graduate students — so teachers can help younger students understand the methods and outcomes of scientific research.

Here’s one of those annotated papers. A widget called the Learning Lens toggles layers of annotation and off.

Here I’ve selected the Glossary layer, and I’ve clicked on the word “distal” to reveal the annotation attached to it.

Now lets look behind the scenes:

Hypothesis was used to annotate the word “distal”. But Learning Lens predated the use of Hypothesis, and the Science in the Classroom team wanted to keep using Learning Lens to display annotations. What they didn’t want was the workflow behind it, which required manual insertion of annotations into HTML pages.

Here’s the solution we came up with. Use Hypothesis to create annotations, then use some JavaScript in Science in the Classroom pages to retrieve Hypothesis annotations and write them into the pages, using the same format that had been applied manually. The preexisting and unmodified Learning Lens JavaScript can then do what it does: pick up the annotations, assign color-coded highlights based on tags, and show the annotations when you click on the highlights.

What made this possible was a JavaScript library that helps with the heavy lifting required to attach an annotation to its intended target in the document.

That library is part of the Hypothesis client, but it’s also available as a standalone module that can be used for other purposes. It’s a nice example of how open source components can enable an ecosystem of interoperable annotation services.

DigiPo / EIC

Next up is a toolkit for student fact-checkers and investigative journalists. You’ve already heard from Mike Caulfield about the Digital Polarization Project, or DigiPo, and from Stefan Candea about the European Investigative Collaborations network. Let’s look at how we’ve woven annotation into their investigative workflows.

These investigations are both written and displayed in a wiki. This is a DigiPo example:

I did the investigation of this claim myself, to test out the process we were developing. It required me to gather a whole lot of supporting evidence before I could begin to analyze the claim. I used a Hypothesis tag to collect annotations related to the investigation, and you can see them in this Hypothesis view:

I can be very disciplined about using tags this way, but it’s a lot to ask of students, or really almost anyone. So we created a tool that knows about the set of investigations underway in the wiki, and offers the names of those pages as selectable tags.

Here I’ve selected a piece of evidence for that investigation. I’m going to annotate it, not by using Hypothesis directly, but instead by using a function in a separate DigiPo extension. That function uses the core anchoring libraries to create annotations in the same way the Hypothesis client does.

But it leads the user through an interstitial page that asks which investigation the annotation belongs to, and assigns a corresponding tag to the annotation it creates.

Back in the wiki, the page embeds the same Hypothesis view we’ve already seen, as a Related Annotations widget pinned to that particular tag:

I had so much raw material for this article that I needed some help organizing it. So I added a Timeline widget that gathers a subset of the source annotations that are tagged with dates.

To put something onto the timeline, you select a date on a page.

Then you create an annotation with a tag corresponding to the date.

Here’s what the annotation looks like in Hypothesis.

Over in the wiki, our JavaScript finds annotations that have these date tags and arranges them on the Timeline.

Publication dates aren’t always evident on web pages, sometimes you have to do some digging to find them. When you do find one, and annotate a page with it, you’ve done more than populate the Timeline in a DigiPo page. That date annotation is now attached to the source page for anyone to discover, using Hypothesis or any other annotation-aware viewer. And that’s true for all the annotations created by DigiPo investigators. They’re woven into DigiPo pages, but they’re also available for separate reuse and aggregation.

The last and most popular annotation-related feature we added to the toolkit is called Footnotes. Once you’ve gathered your raw material into the Related Annotations bucket, and maybe organized some of it onto the Timeline, you’ll want to weave the most pertinent references into the analysis you’re writing.

To do that, you find the annotation you gathered and use Copy to clipboard to capture the direct link.

Then you wrap that link around some text in the article:

When you refresh the page, here’s what you get. The direct link does what a direct link does: it takes you to the page, scrolls you to the annotation in context. But it can take a while to review a bunch of sources that way.

So the page’s JavaScript also creates a link that points down into the Footnotes section. And there, as Ted Nelson would say, and as Nate Angell for some reason hates hearing me say, the footnote is “transcluded” into the page so all the supporting context is right there.

One final point about this toolkit. Students don’t like the writing tools available in wikis, and for good reason, they’re pretty rough around the edges. So we want to enable them to write in Google Docs. We also want them to footnote their articles using direct links because that’s the best way to do it. So here’s a solution we’re trying. From the wiki you’ll launch into Google Docs where you can do your writing in a much more robust editor that makes it really easy to include images and charts. And if you use direct links in that Google Doc, they’ll still show up as Footnotes.

We’re not yet sure this will pan out, but my colleague Maryann Martone, who uses Hypothesis to gather raw material for her scientific papers, and who writes them in Google Docs, would love to be able to flow annotations through her writing tool and into published footnotes.

SciBot

Maryann is the perfect segue to our next example. Along with Anita Bandrowski, she’s working to increase the thread count in the fabric of scientific literature. When neuroscientists write up the methods used in their experiments, the ingredients often include highly specific antibodies. These have colloquial names, and even vendor catalog numbers, but they still lacked unique identifiers. So the Neuroscience Information Framework, NIF for short, has defined a namespace called RRID (research resource identifier), built a registry for RRIDs, and convinced a growing number of authors to mention RRIDs in their papers.

Here’s an article with RRIDs in it. They’re written directly into the text because the text is the scientific record, it’s the only artifact that’s guaranteed to be preserved. So if you’re talking about a goat polyclonal antibody, you look it up in the registery, capture its ID, and write it directly into the text. And if it’s not in the registry, please add it, you’ll make Anita very happy if you do!

The first phase of a project we call SciBot was about validating those RRIDs. They’re just freetext, after all, typed in by authors. Were the identifiers spelled correctly? Did they point to actual registry entries? To find out we built a tool that automatically annotates occurrences of RRIDs.

In this example, Anita is about to click on the SciBot tool, which launches from a bookmarklet, and sends the text of the paper to a backend service. It scans the text for RRIDs, looks up each one in the registry, and uses the Hypothesis API to create an annotation — bound to the occurrence in the text — that reports the results of the registry lookup.

Here the Hypothesis realtime API is showing that SciBot has created three annotations on this page.

And here are those three annotations, anchored to their occurrences in the page, with registry entries displayed in the sidebar.

SciBot curators review these annotations and use tags to mark which are valid. When some aren’t, and need attention, the highlight focuses that attention on a specific occurrence.

This hybrid of automatic entity recognition and interactive human curation is really powerful. Here’s an example where an antibody doesn’t have an RRID but should.

Every automatic workflow needs human exception handling and error correction. Here the curator has marked an RRID that wasn’t written into the literature, but now is present in the annotation layer.

These corrections are now available to train a next-gen entity recognizer. Iterating through that kind of feedback loop will be a powerful way to mine the implicit data that’s woven into the scientific literature and make it explicit.

Here’s the Hypothesis dashboard for one of the SciBot curators. The tag cloud gives you a pretty good sense of how this process has been unfolding so far.

Publishers have begun to link RRIDs to the NIF registry. Here’s an example at PubMed.

If you follow the ZIRC_ZL1 link to the registry, you’ll find a list of other papers whose authors used the same experimental ingredient, which happens to be a particular strain of zebrafish.

This is the main purpose of RRIDs. If that zebrafish is part of my experiment, I want to find who else has used it, and what their experiences have been — not just what they reported in their papers, but ideally also what’s been discussed in the annotation layer.

Of course I can visit those papers, and search within them for ZIRC_ZLI, but with annotations we can do better. In DigiPo we saw how footnoted quotes from source documents can transclude into an article. Publishers could do the same here.

Or we could do this. It’s a little tool that offers to look up an RRID selected in text.

It just links to an instance of the Hypothesis dashboard that’s pinned to the tag for that RRID.

Those search results offer direct links that take you to each occurrence in context.

Claim Chart

Finally, and to bring us full circle, I recently reconnected with Andrew Schulman who works nowadays as a software patent attorney. There’s a tool of his trade called a claim chart. It’s a two-column table. In column one you list claims that a patent is making, which are selections of text from the claims section of the patent. And in column two you assemble bits of evidence, gathered from other sources, that bear on specific claims. Those bits of evidence are selections of text in other documents. It’s tedious to build a claim chart, it involves a lot of copying and pasting, and the evidence you gather is typically trapped in whatever document you create.

Andrew wondered if an annotation-powered app could help build claim charts, and also make the supporting evidence web-addressable for all the reasons we’ve discussed. If I’ve learned anything about annotation, it’s that when somebody asks “Can you do X with annotation?” the answer should always be: “I don’t know, should be possible, let’s find out.”

So, here’s an annotation-powered claim chart.

The daggers at top left in each cell are direct links. The ones in the first column go to patent claims in context.

The ones in the second column go to related statements in other documents.

And here’s how the columns are related. When you annotate a claim, you use a toolkit function called Add Selection as Claim.

Your selection here identifies the target document (that is, the patent), the claim chart you’re building (here, it’s a wiki page called andrew_test), and the claim itself (for example, claim 1).

Once you’ve identified the claims in this way, they’re available as targets of annotations in other documents. From a selection in another document, you use Add Selection as Claim-Related.

Here you see all the claims you’ve marked up, so it’s easy to connect the two statements.

The last time I read Vannevar Bush’s famous essay As We May Think, this was the quote that stuck with me.

When statements in documents become addressable resources on the web, we can weave them together in the way Vannevar Bush imagined.

Do Repeat Yourself, With Variations

Don’t Repeat Yourself (DRY) is a touchstone principle of software development. It’s often understood to inveigh against duplication of code. Copying a half-dozen lines from one program to another is a bad idea, DRY says, because if you change your mind about how that code works, you’ll have to revise it in several places. Better to convert those lines of code into a function that you write once and reuse.

More broadly, the DRY principle asserts:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

Code and data are two kinds of knowledge that ought to be represented canonically, and repeated — if at all — only by mechanical derivation, never by variation.

I often violate the DRY principle by indulging in CopyAndPasteProgramming. In my defense I point to another principle, CodeHarvesting, which defends duplication as a necessary stepping stone.

Letting a duplication of logic live for now, in order to see how to best universalize it at some later point.

For me, at least, that’s what tends to work best. A common theme doesn’t emerge until I’ve seen — and ideally others have seen and reacted to — several variations on that theme. This kind of duplication — deferred universalization — is beneficial, right?

Here’s another kind. In the JavaScript world the dominant engine of reuse is the Node Package Manager (NPM). When I first started using it a few years ago, I was shocked at the amount of duplication it entails. When you install an NPM package, the modules it depends on are copied into a subdirectory. If those modules depend on others, they are copied into yet deeper subdirectories. For even a simple JavaScript program you can end up with a forest of thousands of files.

A similar thing happens in the Python world. It’s a best practice, nowadays, to use a tool called virtualenv to create, for each Python program you run, an isolated environment with the particular Python interpreter and set of modules needed by that particular program. In practice that means, again, copying lots of files.

Arguably these duplications don’t violate DRY because they are mechanical copies that won’t vary from their originals. But they can! And here too I am prone to indulge in local variation to explore possibilities that might or might not warrant generalization.

While pondering the vices and virtues of duplicative software construction I reread Metamagical Themas, the compendium of Douglas Hofstadter’s columns in Scientific American. (The title is an anagram of Mathematical Games, the column he inherited from Martin Gardner.) In Variations on a Theme as the Crux of Creativity he states the case as plainly as anywhere. At the core of creative thought are “slippery” concepts that we develop in a virtuous cycle of innovation:

Once you have decided to try out a new way of viewing a phenomenon, you can let that view suggest a set of knobs to vary. The act of varying them will lead you down new pathways, generating new images ripe for perception in their own right.

This sets up a closed loop:

– fresh situations get unconsciously framed in terms of familiar concept;

– those familiar concepts come equipped with standard knobs to twiddle;

– twiddling those knobs carries you into fresh new conceptual territory.

We need to get DRY eventually in order to maintain stable systems. But the countervailing state needn’t be WET (“write everything twice”, “we enjoy typing” or “waste everyone’s time”). Instead I propose DRYWV: Do Repeat Yourself, With Variations.

Every piece of knowledge should have a single, unambiguous, authoritative representation within a system. But how do we arrive at such knowledge? I think we have to DRYWV our way there.

Dwelling in the zone of evidence

I’ve written plenty about the software layer that adapts the Hypothesis annotator to the needs of someone who gathers, organizes, analyzes, and then writes about evidence found online. Students in courses based on Mike Caulfield’s Digital Polarization template will, I hope, find that this software streamlines the grunt work required to find and cite the evidence that supports evaluation of a claim like this one:

Claim: The North Carolina Republican Party sent out a press release boasting about how its efforts drove down African-American turnout in the 2016 US presidential election.

That’s a lightly-edited version of something I read in the New Yorker and can send you to directly:

As we were fleshing out how a DigiPo course would work, I wrote an analysis of that claim. The investigation took me all the way back to the 1965 Voting Rights Act. Then it led to the 2013 Supreme Court decision — in Shelby vs Holder — to dilute the “strong medicine” Congress had deemed necessary “to address entrenched racial discrimination in voting.” Then to a series of legal contests as North Carolina began adjusting its voting laws. Then to the election-year controversies about voter suppression. And finally to the press release that the North Carolina GOP sent the day before the election, and the reactions to it.

Many claims don’t require this kind of deep dive. As Mike writes today, core strategies — look for fact-checking prior art, go upstream to the source, read laterally — can resolve some claims quickly.

But some claims do require a deep dive. In those cases I want students to immerse themselves in that process of discovery. I want them to suspend judgement about the claim and focus initially on marshalling evidence, evaluating sources, and laying a foundation for analyis. It’s hard work that the DigiPo toolkit can make easier, maybe even fun. That’s crucial because the longer you can comfortably dwell in that zone of evidence-gathering and suspended judgement, the stronger your critical thinking will become.

When I first read Toobin’s claim my internal narrative was: “Boasted about voter suppression? Of course those neanderthals did!” Then I entered the zone and spent many hours there. Voter suppression wasn’t a topic I’d spent much time reading about, so I learned a lot. When I returned to the claim I arrived at an interesting judgement. Yes there was voter suppression, and it was in some ways more draconian than I had thought. But had the North Carolina GOP actually boasted (Mother Jones: bragged, Salon: celebrated) the lower African-American turnout? I concluded it had not. It had reported reduced early voting, but not explicitly claimed that was a successful outcome of voter suppression.

So we rated the claim as Mixed — that is, partly true, partly false. A next step for this investigation would be to break the claim into more granular parts. (Software developers would call that “refactoring” the claim.) So:

In a press release on November 7, 2016, the North Carolina GOP reported lower African-American early voting.

That’s easy to check. True.

Here’s another:

In its 11/7/2016 press release the North Carolina GOP boasted about the success of its voter suppression efforts.

Also easy to check: False.

What about this?

In the wake of Shelby vs Holder, the North Carolina GOP pushed legislation that discriminates against African-American voters.

You need to gather and organize a lot of source material in order to even begin to evaluate that claim. My fondest hope for DigiPo is that students inclined to judge the claim, one way or the another, will delay that judgement long enough to gather evidence that all can agree is valid. That, I believe, would be a fantastic educational outcome.

How shared vocabularies tie the annotated web together

I’m fired up about the work I want to share at Domains 2017 this summer. The tagline for the conference is Indie Tech and Other Curiosities, and I plan to be one of the curiosities!

I’ve long been a cheerleader for the Domain of One’s Own movement. In Reclaiming Innovation, Jim Groom wrote about the need to “understand technologies as ‘potentiality’ (to graft a concept by Anton Chekov from a literary to a technical context).” He continued:

This is the idea that within the use of every technical tool there is more than just the consciousness of that tool, there is also the possibility to spark something beyond those predefined uses. The only real way to galvanize that potentiality is to provide the conditions of possibility — that is, a toolkit for user innovation.

My recent collaboration with Mike Caulfield on the Digital Polarization Initiative has led to the creation of just such a toolkit. It supports DigiPo in the ways described and shown here. A version of the toolkit, demoed here, will support a team of investigative journalists. Now I need to show how the toolkit enables educators, scientists, investigative reporters, students — anyone who researches and writes articles or reports or papers backed by web-based evidence — to innovate in similar ways.

In tech we tend to abuse the term innovation so let me spell out exactly what I mean: Better ways to gather, organize, reason over, and cite online evidence. Web annotation, standardized this week by the W3C, is a key enabler. The web’s infinite space of addressable URLs is now augmented by a larger infinity of segments of interest within the resources pointed to by URLs. In the textual realm, paragraphs, list items, sentences, or individual words can be reliably linked to conversations — but also applications — that live in connected annotation layers.

A web of addressable segments of interest is a necessary, but not sufficient, condition of possibility. We also need tools that enable us to gather, organize, recombine, and cite those segments. And some of those tools need to be malleable in the hands of users who can shape them for their own purposes.

When I reread Vannevar Bush’s As We May Think, to prepare for a conversation about it with Gardner Campbell and Jeremy Dean (video, Gardner’s reflections), I focused on this passage:

He has dozens of possibly pertinent books and articles in his memex. First he runs through an encyclopedia, finds an interesting but sketchy article, leaves it projected. Next, in a history, he finds another pertinent item, and ties the two together.

Nowadays that first encyclopedia article lives at one URL. The pertinent item in a history is a segment of interest within another URL-addressable resource. How do we tie them together? A crucial connector is a tag that belongs to neither resource but refers to both.

When tools control the sets of tags available for resource interconnection, they enable groups of people to make such connections reliably. That’s what the DigiPo toolkit does when it offers a list of investigation pages, drawn from the namespace of a wiki, as the set of tags that connect annotation-defined evidence to investigations. You see that happening with the DigiPo toolkit shown here, and with a variant of the toolkit shown here. In both cases the tags that bind evidence to wiki pages are controlled by software that acquires a list of wiki pages and presents the names of those pages as selectable tags.

One future direction for the toolkit leads to software that acquires lists of pages from other kinds of content management systems: WordPress, Drupal, you name it. Every CMS defines a namespace that is implicitly a list of tags that can be used to bind sets of resources to the pages served by that CMS. If you’re looking to adapt a DigiPo-like tool to your CMS, I’ll be delighted to show you how.

Such adaptation, though, requires somebody to write some code. While it’s unfashionable in some circles to say so, I don’t think everyone should learn to code. There’s a more fundamental web literacy, nicely captured by Audrey Watters here:

It’s about understanding the components of the Web and knowing how to tag and then manipulate them. By thinking and developing sets of named resources, you are a Web thinker. This isn’t about programming but rather the creation of sets of resources and the identification of components that work with those resources and combine them to create solutions.

Web annotation vastly enlarges the universe of resources that can be named. But it’s on us to name them. Tags are a principal way we do that. If our naming of resources is going to be an effective way to organize and combine them, though, we need to do it reliably and consistently. Software can enforce that consistency, but not everyone can write software. So a user innovation toolkit for the annotated web needs to empower users to enforce consistent naming without writing code.

A couple of weeks ago I built a Chrome extension that enables users to define their own lists of shared tags by recording them in an Google Doc. The demonstration video prompted this query from Jim Groom:

I just got through with a workshop here demoing Hypothes.is for a European group that may be using it to annotate online legislation for data privacy set to go live in 2018. They are teaching a course on it, and this could be one of the spaces/hubs they build the open part around. I came back to this video just now, but got the sense I could already tag from within annotations/pages, so how does the tag helper change this? Just a different way at it? Is it new functionality from previous tags? I love that you can have a Google Doc list of tags, but the video example is not making sense to me for some reason. And I wanna know :)

Here’s my response. That tag helper, now incorporated into the toolkit I’m evolving for DigiPo and other uses, makes it possible for people who don’t write code to define tag namespaces that govern their gathering, organization, recombination, and citation not only of URL-addressable resources but also of annotation-addressable segments of interest within those resources. People can “tie them together” — as Vannevar Bush imagined — in the ways their interests and workflows require.

Does that answer the question? If not, please keep asking until I do so properly. User-defined tag namespaces, though admittedly still a curiosity, are one of the best ways to make collective use of a web of addressable segments.

How annotation layers define “segments of interest” for new kinds of applications

Here are some analogies we use when talking about software:

Construction: Programs are houses built on foundations called platforms.

Ecology: Programs are organisms that depend on ecosystem services provided by platforms.

Community: Programs work together in accordance with rules defined by platforms.

Architecture: Programs are planned, designed, and built according to architectural plans.

Economics: Programs are producers and consumers of services.

Computer hardware: Programs are components that attach to a shared bus.

All are valid and may be useful in one way or another. In this essay I focus on the last because it points to an important way of understanding what web annotation can enable. My claim here is that the web’s emerging annotation layer forms a shared bus for a new wave of content-oriented applications.

A computer’s bus connects devices: disk drive, keyboard, network adapter. If we think of the web in this way, we’d say that devices (your computer, mine) and also people (you, me) attach to the bus. And that the protocol for attachment has something to do with URLs.

You can, for example, follow this link to display and interact with the set of Hypothesis annotations related to this web page. You can also paste the link’s URL into a message or a document to share the view with someone else.

That same URL can behave like an API (application programming interface) that accesses the resource named and located by the URL. A page like this one, part of the DigiPo fact-checking project, uses the link that way. It derives the Hypothes search URL from its own URL, and injects the resulting Hypothesis view into the page.

Every time we create a new wiki page at digipo.io, we mint a new URL that summons the set of Hypothesis annotations specific to that page. In principle there’s no limit to the number of such pages — and associated sets of annotations — we can add. And that’s just one of an unlimited number of sites. The web of URL-addressable resources is infinitely large.

Even so, URLs address only a small part of a larger infinity of resources: words and phrases in texts, regions within images, segments of audio and video. Web annotation enables us to address that larger infinity. The DigiPo project illustrates some of the ways in which annotation expands the notion of content as a bus shared by people and computers. But first some background on how annotation works.

The proposed standard for web annotation defines an extensible set of selectors:

Many Annotations refer to part of a resource, rather than all of it, as the Target. We call that part of the resource a Segment (of Interest). A Selector is used to describe how to determine the Segment from within the Source resource.

When the segment of interest is a selection in a textual resource, one kind of selector captures the selection and its surrounding text. Another captures the position of the selection (“starts at the 347th character, ends at the 364th”). Still another captures its location in a web page (“contained in the 2nd list item in the first list in the seventh paragraph”). For reasons of both speed and reliability, Hypothesis uses all three selectors when it attaches (“anchors”) an annotation to a selection.

When a segment of interest is a clip within a podcast or a video, a selector would capture the start and stop (“starts at 1 minute, 32 seconds, ends at 3 minutes, 12 seconds”). When it’s a region in a bitmapped image, a selector would capture the coordinates (“starts at x=12,y=53, ends at x=355,y=124”). When it’s a piece of a vector image, a selector would capture the Scalable Vector Graphics (SVG) markup defining that piece of the image.

The W3C’s model of web annotation lays a foundation for other kinds of selectors in other domains: locations in maps, nodes in Jupyter notebooks, bars and trend lines and data points in charts. But let’s stick with textual annotation for now, consider how it expands the universe of addressable resources, and explore what we can do in that universe.

Here’s a picture of what’s happening in and around the above-mentioned DigiPo page:

The author has cited a Hypothesis link that refers to a piece of evidence in another web page. The link encapsulates both the URL of that page and a set of selectors that mark the selected passage within it. When you follow the link Hypothesis takes you to the page, scrolls to the passage, and highlights it. That’s a powerful interactive experience!

Now suppose you want to review all the evidence that supports this investigation. You can do it interactively but that will require a lot of context-disrupting clicks. So another program embedded in the wiki page summarizes the cited quotes for you. It uses a variant of the Hypothesis direct link that delivers the interactive experience. The variant is a Hypothesis API call that delivers the annotation in a machine-friendly format. The summarization script collects all the Hypothesis direct links on the page, gathers the annotations, extracts the URLs and quotes, injects them into the Footnotes section of the page, and rewrites the links to point to corresponding footnotes.

To enable this magic, an app that people can use to annotate regions in web pages is necessary but not sufficient. You also need an API-accessible service that enables computers to create and retrieve annotations. Even more fundamentally, you need an open web standard that defines how apps and services work not only with atomic resources named and located by URLs, but also segments of interest within them.

What else is possible on a shared content bus where segments of interest are directly addressable both by people and computers? Here’s one idea being pondered by some folks in the world of open educational resources (OER). Suppose you’re creating an open textbook that attaches quizzes to segments within the text. The quizzes live in a database. How do you connect a quiz to a segment in your book?

Because a quiz is an URL-addressable resource, you can transclude one directly into your book near the segment to which it applies. Doing that normally means encoding the segment’s location in the book’s markup so the software that attaches the quiz can put it in the right place. That works, but it entangles two editorial tasks: writing the book, and curating the quizzes. That entanglement makes it harder to provide tools that support the tasks individually. If you can annotate segments of interest, though, you can disentangle the tasks, tool them separately, build the book more efficiently, and ensure others can more cleanly repurpose your work.

Analogies are necessary but imperfect. The notion of a shared bus, formed by an annotation layer and used by applications oriented to segments of content, may or may not resonate. I’m looking for a better analogy; suggestions welcome. But however you want to think about it, the method I’m describing here works powerfully well, I’ll continue to apply it, and I’d love to discuss ways you can too.