How and why to tell your story online, revisited

I wrote this essay in 2006 as part of a series of Internet explainers I did for New Hampshire Public Radio. It never aired for reasons lost to history, so I’m publishing this 15-year-old time capsule here for the first time. My motive is of course not purely archival. I’m also reminding myself why I should still practice now what I preached then.

How and why to tell your story online

Teens and twenty-somethings are flocking to social websites like MySpace and Facebook, where they post photos, music, and personal diaries. Parents, teachers, and cops wish they wouldn’t. It’s a culture war between generations, and right now everybody’s losing.

Kids: Listen up. Did you hear the story about the college student who didn’t get hired because of his Facebook page? Or the teenage girl whose MySpace blog told an attacker when she’d be home alone? These things happen very rarely, but they can happen. Realize that the words and pictures you publish online will follow you around for the rest of your lives. Realize that wrong choices can have embarrassing or even tragic consequences.

Now, grownups, it’s your turn to listen up. You’re right to worry about the kids. But there’s another side to the story. The new forms of Internet self-publishing — including social networks, blogs, podcasting, and video sharing — can be much more than narcissistic games. Properly understood and applied, they’re power tools for claiming identity, exerting influence, and managing reputation. Sadly, very few adults are learning those skills, and fewer still are teaching them.

It’s not enough to condemn bad online behavior. We’ve got to model good online behavior too — in schools, on the job, and in civic life. But we’re stuck in a Catch-22 situation. Kids, who intuit the benefits of the new social media, fail to appreciate the risks. Grownups, meanwhile, see only risks and no benefits.

There’s a middle ground here, and we need to approach it from both sides of the generation gap. The new reality is that, from now on, our lives will be documented online — perhaps by us, perhaps by others, perhaps by both. We may or may not influence what others will say about us. But we can surely control our own narratives, and shape them in ways that advance our personal, educational, professional, and civic agendas.

Your online identity is a lifelong asset. If you invest in it foolishly you’ll regret that. But failing to invest at all is equally foolish. The best strategy, as always, is to invest wisely.

Here’s a simple test to guide your strategy. Imagine someone searching Google for your name. That person might be a college admissions officer, a prospective employer, a new client, an old friend, or even a complete stranger. The reason for the search might be to evaluate your knowledge, interests, agenda, accomplishments, credentials, activities, or reputation.

What do you want that person to find? That’s what you should publish online.

To find three examples of what I mean, try searching the web for the following three names: Todd Suomela, Martha Burtis, Thomas Mahon. In each case, the first Google result points to a personal blog that narrates a professional life.

Todd Suomela is a graduate student at the University of Michigan. On his blog, Todd writes about what he’s learning, and about how his interests and goals are evolving. He hasn’t launched his professional career yet. But when he does, his habit of sharing the information resources he collects, and reflecting thoughtfully on his educational experience, will serve him well.

Martha Burtis is an instructional technologist at the University of Mary Washington. She and her team research and deploy the technologies that students, faculty, and staff use to learn, teach, and collaborate. On her blog, Martha writes about the tools and techniques she and her team are developing, she assesses how her local academic community is making use of those tools and techniques, and thinks broadly about the future of education.

Thomas Mahon is a Savile Row tailor. His shop in London caters to people who can spend two thousand pounds on a classic handmade suit. I’ll never be in the market for one of those, but if I were I’d be fascinated by Mahon’s blog,, which tells you everything you might want to know about Savile Row past and present, about how Mahan practices the craft of bespoke tailoring, and about how to buy and care for the garments he makes.

For Todd and Martha and Thomas, the benefits of claiming their Net identities in these ways run wide and deep. Over time, their online narratives become autobiographies read by friends, colleagues, or clients, and just as importantly, read by people who may one day become friends, colleagues, or clients.

In most cases, of course, the words, pictures, audio, and video you might choose to publish online won’t attract many readers, listeners, or viewers. That’s OK. The point is that the people they do attract will be exactly the right people: those who share your interests and goals.

We’ve always used the term ‘social networking’ to refer to the process of finding and connecting with those people. And that process has always depended on a fabric of trust woven most easily in the context of local communities and face-to-face interaction.

But our interests and goals aren’t merely local. We face global challenges that compel us to collaborate on a global scale. Luckily, the new modes of social networking can reach across the Internet to include people anywhere and everywhere. But if we’re going to trust people across the Internet, we’ll need to be able check their references. Self-published narrative is one crucial form of evidence. The public reaction to such narratives, readily discoverable thanks to search engines and citation indexes, is another.

Is this a new and strange new activity? From one perspective it is, and that’s why I can’t yet point to many other folks who’ve figured out appropriate and effective ways to be online, as Todd and Martha and Thomas have.

But from another perspective, Internet self-publishing is just a new way to do what we’ve been doing for tens of thousand years: telling stories to explain ourselves to one another, and to make sense of our world.

The Image of Postgres

At the 2015 Postgres conference, the great IT philosopher Robert r0ml Lefkowitz delivered a talk entitled The Image of Postgres. Here’s the blurb.

How do you think about what a database is? If you think of a database as only a place to store your data, then perhaps it does not really matter what the internals of that database are; all you really need is a home for your data to be managed, nothing more.

If you think of a database as a place where you develop applications, then your expectations of your database software change. No longer do you only need data management capabilities, but you require processing functions, the ability to load in additional libraries, interface with other databases, and perhaps even additional language support.

If your database is just for storage, there are plenty of options. If your database is your development framework, you need Postgres.

Why? Well, let’s get philosophical.

For over a year, I’ve been using Postgres as a development framework. In addition to the core Postgres server that stores all the Hypothesis user, group, and annotation data, there’s now also a separate Postgres server that provides an interpretive layer on top of the raw data. It synthesizes and caches product- and business-relevant views, using a combination of PL/pgSQL and PL/Python. Data and business logic share a common environment. Although I didn’t make the connection until I watched r0ml’s talk, this setup harkens back to the 1980s when Smalltalk (and Lisp, and APL) were programming environments with built-in persistence. The “image” in r0ml’s title refers to the Smalltalk image, i.e. the contents of the Smalltalk virtual machine. It may also connote reputation, in the sense that our image of Postgres isn’t that of a Smalltalk-like environment, though r0ml thinks it should be, and my experience so far leads me to agree.

I started writing a book to document what I’ve learned and done with this idea. It’s been a struggle to find motivation because, well, being the patron saint of trailing-edge technologies is often lonely and unrewarding. A book on this particular topic is likely to appeal to very few people. Stored procedures? So last century! Yes, Python provides a modern hook, but I can almost guarantee that one of the comments on my first book — “has a vision, but too weird” — would come back around.

I’m tempted not to bother. Maybe I should just focus on completing and polishing the things the book would describe.

And yet, it’s hard to let go. This isn’t just a compelling idea, it’s delivering excellent results. I rewatched r0ml’s talk today and got fired up again. Does it resonate for you? Would you like to see the ideas developed? If you watch the talk, please let me know.

Here are some excerpts to pique your interest.

On databases vs file systems:

I submit that the difference between the database and a file system is that database is a platform for enforcing your business rules.

On ontology:

client: The business guys are always upset because they want to know how many customers we have and we can’t tell them.

r0ml: That doesn’t sound like a very hard problem. SELECT * from the customer table, right?

client: No you don’t understand the problem.

r0ml: OK, what’s the problem?

client: It depends what you mean by customer because if you’re selling cell phone insurance, is the customer the person who has the cell phone account? What if they have two handsets and they’re both insured? What if it’s a family account and there are kids on the plan, do they count as customers? What if it’s a business account and you have 1000 people covered but only 700 using?

r0ml: How my customers you have, that’s a business rule.

So figuring out what your schema is, and figuring out how you organize the stuff and what do you do in the database, that’s all part of enforcing your business rules.

You have to decide what these things mean.

It’s an ontological problem.

You have to classify your knowledge and then enforce your business rules.

On n-tier architecture:

Let us think about the typical web application architecture. This architecture is called the three-tier architecture because it has four tiers. You have your browser, your web server, the thing that runs Python or PHP or JavaScript or Ruby or Java code, and then the database. And that’s always how you do it. And why do you do it that way? Well because that’s how everybody does it.

On Smalltalk and friends:

This is the BYTE magazine cover from August of 1981. In the 70s and the 80s, programming languages had this sort of unique perspective that’s completely lost to history. The way it worked: a programming environment was a virtual machine image, it was a complete copy of your entire virtual machine memory and that was called the image. And then you loaded that up and it had all your functions and your data in it, and then you ran that for a while until you were sort of done and then you saved it out. And this wasn’t just Smalltalk, Lisp worked that way, APL worked that way, it was kind of like Docker only it wasn’t a separate thing because everything worked that way and so you didn’t worry very much about persistence because it was implied. If you had a programming environment it saved everything that you were doing in the programming environment, you didn’t have to separate that part out. A programming environment was a place where you kept all your data and business logic forever.

So then Postgres is kind of like Smalltalk only different.

What’s the difference? Well we took the UI out of Smalltalk and put it in the browser. The rest of it is the same, so really Postgres is an application delivery platform, just like we had back in the 80s.

Why public phones still exist

My superpower has always been finding new uses for old tech. In the late 90s I dusted off the venerable NNTP server, which had been the backbone of the Usenet, and turned it into my team’s Slack. In the late 2000s I used iCalendar to make citywide event calendars. In the late 2010s I went deep into SQL.

It’s always intensely pragmatic. But also, I can’t deny, whimsical.

In that spirit, I offer you the public pay phone at the Pinnacles Visitor Center. I stayed in that campground on a road trip just before the election. Given the tense political and epidemiological situation, I’d promised to message home regularly. There was no cell service in the park so I headed over to the office. It was closed, so I sat on the bench and connected to their WiFi. Or tried to. You could connect, sometimes, but you couldn’t move any data. The router was clearly in need of a reboot.

The only option left was the public phone. I can’t remember the last time I used one. Most people alive today have, perhaps, never used one. But there it was, so I gave it a shot.

Once upon a time, you could pick up the handset, dial 0 for operator, and place a so-called collect (charge-reversed) call. Now dialing 0 gets you nowhere.

The instructions taped to the phone (in the 90s I’m guessing) say you can call an 800 number, or use a calling card. I remember calling cards, I had one once. Not a thing lately.

And then there was this: “Dial 611 for help.”

Me: 611

611: Hello, this is Steve.

Me: I’m at the Pinnacles Visitor Center trying to send a message.

Steve: Use the WiFi.

Me: I can’t, it’s broken.

Steve: Huh, that’s interesting. Let me see if I can reboot the router.

And he did. So there you have it. The public phone still provides a valuable service. Its mission has evolved over the years. Nowadays, it exists to summon Steve the IT guy who can fix the WiFi by turning it off and on again.

Works like a charm!

Moonstone Beach Breakdown

Travel always has its ups and downs but I don’t think I’ve ever experienced both at the same time as intensely as right now.

I’m at Moonstone Beach in Cambria, just south of San Simeon, in a rented camper van. After a walk on the beach I hop in, reverse, clip a rock, blow a tire, and come to rest alongside the guard rail facing the ocean.

I call roadside assistance; they can deliver a tire but not until tomorrow morning.

I may be about to win the road trip breakdown lottery. I’m snuggled in my two-sleeping-bag nest on the air mattress in the back of the van, on a bluff about 25 feet above the beach, with the van’s big side door open, watching and hearing the tide roll in.

The worst and best parts of my trip are happening at the same time. I screwed up, am stuck, cannot go anywhere. But of all the places I could have been stuck on this trip, I’m stuck in the place I most want to be.

The sign says the gate closes at 6, but nobody has shown up by 7 when everyone else is gone. I can’t reach the authorities. This would be the campsite of my dreams if I’m allowed to stay.

The suspense is killing me.

Eventually a cop shows up, agrees that I can’t go anywhere, and gives me permission to stay for the night. I win the lottery! Nobody ever gets to stay here overnight. But here I am.

We’re all stuck in many ways for many reasons. A road trip during the final week before the election seemed like a way to silence the demons. Roaming around the state didn’t really help. But this night on the bluff over Moonstone Beach most certainly will.

In the light of the full moon, the crests of the waves are sometimes curls of silver, sometimes wraiths of foam that drift slowly south, continually morphing.

I don’t know how we’re all going to get through this winter. I don’t know what comes next. I don’t even have a plan for tomorrow. But I am so grateful to be here now.

Learning analytics for annotated courses

When teachers and students use Hypothesis annotation to explore course readings, certain benefits are immediately obvious. When students find one another in the margins of documents they tell us that the experience of reading is less lonely, which matters now more than ever. When teachers and students discuss key passages, their conversation — linked directly to highlights, displayed in the margin — is more focused than in conventional forums.

This experience of social reading can be delivered as an assignment in a learning management system (LMS). Students are expected to read a chapter of a book and make a prescribed number of substantive annotations; teachers grade the exercise.

Or it can happen less formally. A teacher, who may or may not operate in the context of an LMS, can use annotation to open a window into students’ minds, whether or not their annotations influence their grades.

All these scenarios produce data. The annotated corpus has, at its core, a set of documents. Annotation data explicitly records highlighted passages in documents along with the notes linked to those highlights. It also records conversations that flow from those annotated highlights

Teachers and students, as well as authors and publishers of course readings, can all learn useful things from this data. Teachers and students may want to know who has participated and how often, which are the most highlighted passages across all documents in the course, or which highlights have attracted the longest conversation threads. Authors and publishers will also want to know which passages have attracted the most highlights and discussion.

The annotation data can also support deeper analysis. How often do teachers or students ask questions? How often are questions answered, or not? How often do teachers ask questions answered by students, students ask questions answered by teachers, or students ask questions answered by students?

Analysis of questions and answers

We are providing these views now, on an experimental basis, in order to explore what learning analytics can become in the realm of annotated course reading. Surfacing questions asked by students and answered by students, for example, could be a way to measure the sort of peer interaction that research suggests can improve engagement and outcomes.

Of course the devil is always in the details. Our initial naive approach looks for question marks in annotations, then correlates responses to those annotations. This is convenient because it works with natural discourse, but imprecise because questions are often rhetorical.

A less convenient but more precise approach would require participants to signal intent using an inline hashtag, or a formal tag on the annotation.

A convenient and precise approach would bake affordances into the annotation tool. For example, there might be reaction icons to say things like:

– I’m asking a question that expects an answer

– I’m confused on this point

– I’m highlighting an example of a rhetorical device

In my view we can’t know, a priori, what those baked-in affordances should be. We’ll need to discover them in collaboration with teachers who are using Hypothesis in their courses, and who are willing and able to explore these ideas before they can be fully codified in software.

If you’re a teacher using Hypothesis actively this term, and you’d like to participate in this research, please let us know. We’ll invite you to try our prototype analytics system and help us evolve it.

A good way to encourage voters

I tried to sign up for phone banking and I just couldn’t see doing it, I don’t answer unknown calls and I wouldn’t expect anyone else to right now. I wound up with the Vote Forward letter-writing system,, which I like for a variety of reasons.

It’s really well organized. You get batches of five or 20 letters that are for voters who are likely to be under-represented, and/or have not voted recently. The templates just ask the person to vote, not specifying for whom, and provide links to localized voter info.

They also leave space for you to hand-address the recipient, add a handwritten message, and sign your name.

The last couple of batches I prepared are heading to South Carolina. The letters won’t go out until October 17, though, for maximum impact. This was a compromise, the original plan — backed by research — was for letters to arrive closer to the election. But now that the threat model includes USPS sabotage, the date had to be moved earlier.

Vote Forward claims to have evidence showing that this method makes a difference on the margin. I haven’t seen that evidence, and would like to, but it seems plausible. The recipients are getting a hand-addressed stamped envelope with a return address that has my name atop an address in their region, which is slightly deceptive but feels like a good way to get them to open the letter.

You buy your stamps, so there’s a bit of financial contribution that way.

As a bonus I am re-learning cursive handwriting, which I forgot was primarily about speed. My fifth grade teacher, Mrs. Cloud, who tormented me about my handwriting, would approve.

I’m finding the process to be therapeutic, and it was a much better way to spend an hour and a half the other night than watching the so-called debate.

There’s plenty of time until October 17, so if you’ve been looking for a way to do something and haven’t found it yet, I can recommend this one. I’ve never done anything like this before, I hope the fact I am doing it now is representative of what a lot of others are doing in a lot of ways.

Robert Plomin on heritability

This post is just a record of the key insights I took away from Sam Harris’ enlightening talk with Robert Plomin.

I sure wish it were easier to capture and embed these kinds of audio segments, and to do so more effectively — ideally with transcription. It was way too much work to construct the URLs that deliver these segments into the embedded players I’ve included here, even with the tool I made for that purpose. It’s nice to have a standard embedded player, finally, but since it doesn’t show the endpoints of the segments you can’t tell they’re each just a few minutes. It looks like they all run to the end.

I wound up going to more trouble than it’s probably worth to convey the most memorable parts of that excellent podcast, and I don’t have time to transcribe more fully, but for what it’s worth, here are the segments that I’ll be remembering.

1. “The most important thing we’ve learned from the DNA revolution in the last 10 years is that genetic influences on complex traits are due to thousands of tiny DNA differences.”

2. “These polygenic scores are perfectly normally distributed.”

3. “I’m saying there are no disorders, there are just quantitative dimensions.”

How you can help build the fever map

Rich Kilmer is an old friend who runs CargoSense, a logistics company that gathers and analyzes data from sensors attached to products moving through supply chains. As the pandemic emerged he realized that one particular kind of sensor — the household thermometer used to measure human body temperature — was an underutilized source of crucial data. So the CargoSense team built TrackMyTemp, a web app you can use to provide a stream of anonymized temperature readings to the research community.

It’s dead simple to use. When you first visit the site it asks for your age and current temperature, invites you to activate a virtual thermometer, and prompts for the thermometer’s type (oral, forehead, ear). You also have to allow access to your location so your data can be geocoded.

After you report your first reading, you land on your “personal virtual thermometer” page which you’re invited to bookmark. The URL of that page encodes an anonymous identity. You revisit it whenever you want to contribute a new temperature reading — ideally always at the same time of day.

The precision of your location varies, Rich told me, to account for differences in population density. In a less populous area, an exact location could be a personally identifying signal, so the service fuzzes the location.

Why participate? The site explains:

An elevated temperature can be an indicator that your body is fighting off an infection. Some people contract COVID-19 but never know they have it, because other than a minor increase in temperature, they never show any other symptoms. As we isolate ourselves in our homes for physical distancing purposes we wonder what else we can do. Using this site is something we all can do to help epidemiologists better model how the virus is spreading. By copying your temperature from your physical thermometer into a virtual thermometer using this site, you will help to build a community, and national and global real-time datasets that will help researchers track and combat the spread of COVID-19. We do this while maintaining your privacy, and you only need your mobile phone and your existing thermometer to participate.

You may have seen reports about the Kinsa smart thermometer fever map, which seems to show fevers dropping as social distancing takes hold. This kind of data can help us monitor the initial wave of COVID-19 outbreaks, and could be a critical early warning system later this year as a second wave begins to emerge.

With TrackMyTemp you don’t need a smart thermometer, you can use the ordinary one you already have. I plan to keep one in my office, take a reading when I start my work day, and send the number to TrackMyTemp. If even a tiny fraction of the 128 million U.S. households got into the habit of doing this, we’d produce an ongoing stream of data useful to health researchers.

The best motivation, of course, is enlightened self-interest. So here’s mine. Until this week I can’t remember the last time I took my temperature. Knowing it on a daily basis could help me detect the onset of the virus and amp up my efforts to protect others from it.

Controlling a browser extension with puppeteer

Web technology, at its best, has always occupied a sweet spot at the intersection of two complementary modes: interactive and programmatic. You interact with a web page by plugging its URL into a browser. Then you read what’s there or, if the page is an app, you use the app. Give that same URL, web software can read the page, analyze it, extract data from it, interact with the page, and weave its data and behavior into new apps. Search is the classic illustration of this happy synergy. A crawler reads and navigates the web using the same affordances available to people, then produces data that powers a search engine.

In the early days of the web, it was easy to automate the reading, navigation, analysis, and recombination of web pages because the pages were mostly just text and links. Such automation was (and still is) useful for analysis and recombination, as well as for end-to-end testing of web software. But things got harder as the web moved away from static server-generated pages and toward dynamic client-generated pages made with JavaScript.

All along the way, we’ve had another powerful tool in our kit. Web pages and apps, however they’re produced, can be enhanced with extra JavaScript that’s injected into them. My first experience of the power of that approach was a bookmarklet that, when activated on an Amazon page, checked your local library to see if the book was available there. Then came Greasemonkey, a framework for injecting “userscripts” into pages. It was the forerunner of browser extensions now converging on the WebExtensions API.

The Hypothesis web annotator is a good example of the kind of software that relies on the ability to inject JavaScript into web pages. It runs in two places. The annotator is injected into the web page you’re annotating; the sidebar (i.e. the app) runs in an iframe that’s also injected into the page. The annotator acquires page metadata, reacts to the selection event, forms W3C-style selectors that define the selection, and sends that information (by way of JavaScript messages) to the sidebar. The sidebar queries the server, retrieves annotations for the page into which the annotator was injected, displays the annotations, tells the annotator about navigation it should sync with, and listens for messages from the annotator announcing selections that serve as the basis of new annotations. When you create a new annotation in the sidebar, it messages the annotator so it can anchor it. That means: receive annotation data from the sidebar, and highlight the selection it refers to.

Early in my Hypothesis tenure we needed to deploy a major upgrade to the anchoring machinery. Would annotations created using the old system still anchor when using the new one? You could check interactively by loading HTML and PDF documents into two versions of the extension, then observing and counting the annotations that anchored or didn’t, but that was tedious and not easily repeatable. So I made a tool to automate that manual testing. At the time the automation technology of choice was Selenium WebDriver. I was able to use it well enough to validate the new anchoring system, which we then shipped.

Now there’s another upgrade in the works. We’re switching from a legacy version of PDF.js, the Mozilla viewer that Hypothesis uses to convert PDF files into web pages it can annotate, to the current PDF.js. Again we faced an automation challenge. This time around I used the newer automation technology that is the subject of this post. The ingredients were: headless Chrome, a browser mode that’s open to automation; puppeteer, an API for headless Chrome; and the devtools protocol, which is how the Chromium debugger, puppeteer, and other tools and APIs control the browser.

This approach was more powerful and effective than the Selenium-based one because puppeteer’s devtools-based API affords more intimate control of the browser. Anything you can do interactively in the browser’s debugger console — and that’s a lot! — you can do programmatically by way of the devtools protocol.

That second-generation test harness is now, happily, obsolete thanks to an improved (also puppeteer-based) version. Meanwhile I’ve used what I learned on a couple of other projects. What I’ll describe here, available in this GitHub repo, is a simplified version of one of them. It illustrates how to use puppeteer to load an URL into an extension, simulate user input and action (e.g., clicking a button), open a new extension-hosted tab in reaction to the click, and transmit data gathered from the original tab to the new tab. All this is wrapped in a loop that visits many URLs, gathers analysis results for each, combines the results, and saves them.

This is specialized pattern, to be sure, but I’ve found important uses for it and expect there will be more.

To motivate the example I’ve made an extension that, while admittedly concocted, is plausibly useful as a standalone tool. When active it adds a button labeled view link elements to each page you visit. When you click the button, the extension gathers all the link elements on the page, organizes them, and sends the data to a new tab where you view it. There’s a menagerie of these elements running around on the web, and it can be instructive to reveal them on any given page.

The most concocted aspect of the demo is the input box that accompanies the view link elements button. It’s not strictly required for this tool, but I’ve added it because the ability to automate user input is broadly useful and I want to show how that can work.

The extension itself, in the repo’s browser-extension folder, may serve as a useful example. While minimal, it illustrates some key idioms: injecting code into the current tab; using shadow DOM to isolate injected content from the page’s CSS; sending messages through the extension’s background script in order to create a new tab. To run it, clone the repo (or just download the extension’s handful of files into a folder) and follow these instructions from

  1. Open the Extension Management page by navigating to chrome://extensions. The Extension Management page can also be opened by clicking on the Chrome menu, hovering over More Tools then selecting Extensions.

  2. Enable Developer Mode by clicking the toggle switch next to Developer mode.

  3. Click the LOAD UNPACKED button and select the extension directory.

This works in Chromium-based browsers including Chrome, Brave, and the new version of Edge. The analogous procedure for Firefox:

Open “about:debugging” in Firefox, click “Load Temporary Add-on” and select any file in your extension’s directory.

The Chromium and Firefox extension machinery is not fully compatible. That’s why Hypothesis has yet to ship a Firefox version of our extension, and why the demo extension I’m describing here almost works in Firefox but fails for a reason I’ve yet to debug. Better convergence would of course be ideal. Meanwhile I’m confident that, with some elbow grease, extensions can work in both environments. Chromium’s expanding footprint does tend to erode the motivation to make that happen, but that’s a topic for another day. The scope of what’s possible with Chromium-based browsers and puppeteer is vast, and that’s my topic here.

Of particular interest to me is the notion of automated testing of workflow. The demo extension hints at this idea. It requires user input; the automation satisfies that requirement by providing it. In a real workflow, one that’s driven by a set of rules, this kind of automation can exercise web software in a way that complements unit testing with end-to-end testing in real scenarios. Because the software being automated and the software doing the automation share intimate access to the browser, the synergy between them is powerful in ways I’m not sure I can fully explain. But it makes my spidey sense tingle and I’ve learned to pay attention when that happens.

Here’s a screencast of the runnning automation:

The results gathered from a selection of sites are here.

Some things I noticed and learned along the way:

Languages. Although puppeteer is written in JavaScript, there’s no reason it has to be. Software written in any language could communicate with the browser using the devtools protocol. The fact that puppeteer is written in JavaScript introduces potential confusion in places like this:

const results = await viewPage.evaluate((message) => {
  // this blocks run in the browser's 2nd tab
  const data = JSON.parse(document.querySelector('pre').innerText) 

The first line of JS runs in node, the second line runs in the browser. If puppeteer were written in a different language that would be obvious; since it isn’t I remind myself with a comment.

Settings. If you need the automated instance of Chromium to remember login info or other things, you can add userdir: {PATH} to the object passed to puppeteer.launch.

Interactivity. It may seem odd that the demo is using headless Chrome but specifies headless: false in the object passed to puppeteer.launch. I do that in order to be able to watch the automation when running locally; I’d turn it off when running automation in the cloud. Although I’ve not yet explored the idea, an interactive/automated hybrid seems an intriguing foundation for workflow apps and other kinds of guided experiences.

Web components used in the ClinGen app

A few years back I watched Joe Gregorio’s 2015 OSCON talk and was deeply impressed by the core message: “Stop using JS frameworks, start writing reusable, orthogonally-composable units of HTML+CSS+JS.” Earlier this year I asked Joe to reflect on what he’s since learned about applying the newish suite of web standards known collectively as web components. He replied with a blog post that convinced me to try following in his footsteps.

I’ve been thinking about how software components can be built and wired together since the dawn of the web, when the state-of-the-art web components were Java applets and Netscape browser plugins. Since then we’ve seen newer plugin mechanisms (Flash, Silverlight) rise and fall, and the subsequent rise of JavaScript frameworks (jQuery, Angular, React) that are now the dominant way to package, reuse, and recombine pieces of web functionality.

The framework-centric approach has never seemed right to me. My touchstone example is the tag editor in the Hypothesis web annotator. When the Hypothesis project started, Angular was the then-fashionable framework. The project adopted a component, ngTagsInput, which implements the capable and attractive tag editor you see in Hypothesis today. But Angular has fallen from grace, and the Hypothesis project is migrating to React — more specifically, to the React-API-compatible preact. Unfortunately ngTagsInput can’t come along for the ride. We’ll either have to find a React-compatible replacement or build our own tag-editing component.

Why should a web component be hardwired to an ephemeral JavaScript framework? Why can’t the web platform support framework-agnostic components, as Joe Gregorio suggests?

I’ve been exploring these questions, and my provisional answers are: “It shouldn’t!” and “It can!”

The demo I’ll describe here comes from an app used by biocurators working for the ClinGen Consortium. They annotate scientific literature to provide data that will feed machine learning systems. While these biocurators can and do use off-the-shelf Hypothesis to highlight words and phrases in papers, and link them to search results in databases of phenotypes, genes, and alleles, the workflow is complex and the requirements for accurate tagging are strict. The app provides a guided workflow, and assigns tags automatically. In the latest iteration of the app, we’ve added support for a requirement to tag each piece of evidence with a category — individual, family, or group — and an instance number from one to twenty.

The demo extracts this capability from the app. It’s live here, or you can just download index.html and index.js and open index.html locally. This is plain vanilla web software: no package.json, no build script, no transpiler, no framework.

To create a widget that enables curators to declare they are annotating in the context of individual 1 or family 3 or group 11, I started with IntegerSelect, a custom element that supports markup like this:

  <select is="integer-select" type="individual" count="20"></select>  

Here I digress to clarify a point of terminology. If you look up web components on Wikipedia (or anywhere else) you will learn that the term refers to a suite of web technologies, primarily Custom Elements, Shadow DOM, and HTML Template. What I’m showing here is only Custom Elements because that’s all this project (and others I’m working on) have required.

(People sometimes ask: “Why didn’t web components take off?” Perhaps, in part, because nearly all the literature implies that you must adopt and integrate a wider variety of stuff than may be required.)

The thing being extended by IntegerSelect is a native HTML element, HTMLSelectElement. When it renders into the DOM, it will have empty select markup which the element’s code then fills. It inherits from HTMLSelectElement, so that code can use expressions like this.options[this.selectedIndex].value.

Two key patterns for custom elements are:

  1. Use attributes to pass data into elements.

  2. Use messages to send data out of elements.

You can see both patterns in IntegerSelect’s class definition:

class IntegerSelect extends HTMLSelectElement {
  constructor() {
    this.type = this.getAttribute('type')
  relaySelection() {
    const e = new CustomEvent('labeled-integer-select-event', { 
      detail: {
        type: this.type,
        value: this.options[this.selectedIndex].value
    // if in a collection, tell the collection so it can update its display
    const closestCollection = this.closest('labeled-integer-select-collection')
    if (closestCollection) {
    // tell the app so it can save/restore the collection's state
  connectedCallback() {
    const count = parseInt(this.getAttribute('count'))
    let options = ''
    const lookupInstance = parseInt(getLookupInstance(this.type))
    for (let i = 1; i < count; i++) {
      let selected = ( i == lookupInstance ) ? 'selected' : ''
      options += `<option ${selected}>${i}`
    this.innerHTML = options
    this.onclick = this.relaySelection
customElements.define('integer-select', IntegerSelect, { extends: "select" })

As you can see in the first part of the live demo, this element works as a standalone tag that presents a picklist of count choices associated with the given type. When clicked, it creates a message with a payload like this:


And it beams that message at a couple of targets. The first is an enclosing element that wraps the full hierarchy of elements that compose the widget shown in this demo. The second is the single-page app that hosts the widget. In part 1 of the demo there is no enclosing element, but the app is listening for the event and reports it to the browser’s console.

An earlier iteration relied on a set of radio buttons to capture the type (individual/family/group), and combined that with the instance number from the picklist to produce results like individual 1 and group 3. Later I realized that just clicking on an IntegerSelect picklist conveys all the necessary data. An initial click identifies the picklist’s type and its current instance number. A subsequent click during navigation of the picklist changes the instance number.

That led to a further realization. The radio buttons had been doing two things: identifying the selected type, and providing a label for the selected type. To bring back the labels I wrapped IntegerSelect in LabeledIntegerSelect whose tag looks like this:

  <labeled-integer-select type="individual" count="20"></labeled-integer-select>

Why not a div (or other) tag with is="labeled-integer-select"? Because this custom element doesn’t extend a particular kind of HTML element, like HTMLSelectElement. As such it’s defined a bit differently. IntegerSelect begins like so:

class IntegerSelect extends HTMLSelectElement

And concludes thusly:

customElements.define('integer-select', IntegerSelect, { extends: "select" })

LabeledIntegerSelect, by contrast, begins:

class LabeledIntegerSelect extends HTMLElement

And ends:

customElements.define('labeled-integer-select', LabeledIntegerSelect)

As you can see in part 2 of the demo, the LabeledIntegerSelect picklist is an IntegerSelect picklist with a label. Because it includes an IntegerSelect, it sends the same message when clicked. There is still no enclosing widget to receive that message, but because the app is listening for the message, it is again reported to the console.

Finally we arrive at LabeledIntegerSelectCollection, whose markup looks like this:

    <labeled-integer-select type="individual" count="20"></labeled-integer-select>
    <labeled-integer-select type="family" count="20"></labeled-integer-select>
    <labeled-integer-select type="group" count="20"></labeled-integer-select>

When this element is first connected to the DOM it visits the first of its LabeledIntegerSelects, sets its selected attribute true, and bolds its label. When one of its LabeledIntegerSelects is clicked, it clears that state and reestablishes it for the clicked element.

The LabeledIntegerSelectCollection could also save and restore that state when the page reloads. In the current iteration it doesn’t, though. Instead it offloads that responsibility to the app, which has logic for saving to and restoring from the browser’s localStorage.

You can see the bolding, saving, and restoring of choices in this screencast:

I’m still getting the hang of custom elements, but the more I work with them the more I like them. Joe Gregorio’s vision of “reusable, orthogonally-composable units of HTML+CSS+JS” doesn’t seem far-fetched at all. I’ll leave it to others to debate the merits of the frameworks, toolchains, and package managers that so dramatically complicate modern web development. Meanwhile I’ll do all that I can with the basic web platform, and with custom elements in particular. Larry Wall has always said of Perl, which enabled my earliest web development work, that it makes easy things easy and hard things possible. I want the web platform to earn that same distinction, and I’m more hopeful than ever that it can.

Products and Capabilities

I’ve come to depend on the outlining that’s baked into Visual Studio Code. So far as I can tell, it is unrelated to the tool’s impressive language intelligence. There’s no parsing or understanding of the text in which sections can fold and unfold, but indentation is significant and you can exploit that to make nested expand/collapse zones in any language. I use it for HTML, CSS, JavaScript, Python, Markdown, SQL, and maybe a few others I’m forgetting.

For a long time I thought of myself as someone who respected outlining tools as an important category of software but, having tried various of them going all the way back to ThinkTank, never adopted one as part of my regular work. My experience with VSCode, though, has shown me that while I never adopted an outliner product, I have most surely adopted outlining capability. I use it wherever it manifests. I make tables of contents in Google Docs, I use code folding in text editors that support it, I exploit any outlining affordance that exists in any tool I’m using. Recently I was thrilled to learn that the web platform offers such an affordance in the form of the <details> tag.

In thinking about this post, the first title that occurred to me was: Products vs. Capabilities. Were I still earning a living writing about tech I might have gone with that, or an editor might have imposed it, because conflict (in theory) sells content. But since I’m not selling content it’s my call. I choose Products and Capabilities because the two are, or anyway can (and arguably should) be complementary. For some people, an outliner is a product you use regularly because you value that way of visualizing and reorganizing stuff that you write. You might not be one of those people. I’m not. I don’t, for example, use outlining to organize my thoughts when I write prose, as I’m doing now. That just isn’t how my mind works. I’ve done enough decent writing over the years to accept that it’s OK to not explicitly outline my prose.

For the various kinds of code I spend a lot of time wrangling these days, though, outlining has become indispensable. An outlining expert might say that the capability I’m talking about isn’t real full-fledged outlining, and I’d agree. In a dedicated outliner, for example, you expect to be able to drag things up and down, and in and out, without messing them up. VSCode doesn’t have that, it’s “just” basic indentation-driven folding and unfolding. But that’s useful. And because it shows up everywhere, I’m more effective in a variety of contexts.

In web annotation, I see the same kind of product/capability synergy. Hypothesis is a product that you can adopt and use as a primary tool. You might do that if you’re the sort of person who fell in love with delicious, then moved on to Diigo or Pinboard. Or if you’re a teacher who wants to make students’ reading visible. But web annotation, like outlining, is also a capability that can manifest in — and enhance — other tools. I showed some examples in this 2017 talk; I’ve helped develop others since; I’m more excited than ever about what’s possible.

I’m sure there are other examples. What are some of them? Is there a name for this pattern?

Digital sweatshops then and now

In the early 1960s my family lived in New Delhi. In 1993, working for BYTE, I returned to India to investigate its software industry. The story I wrote, which appears below, includes this vignette:

Rolta does facilities mapping for a U.S. telephone company through its subsidiary in Huntsville, Alabama. Every night, scanned maps flow through the satellite link to Bombay. Operators running 386-based RoltaStations retrieve the maps from a Unix server, digitize them using Intergraph’s MicroStation CAD software, and relay the converted files back to Huntsville.

I didn’t describe the roomful of people sitting at those 386-based RoltaStations doing the work. It was a digital sweatshop. From the window of that air-conditioned room, though, you could watch physical laborers enacting scenes not unlike this one photographed by my dad in 1961.

I don’t have a picture of those rows of map digitizers in the city soon to be renamed Mumbai. But here’s a similar picture from a recent Times story, A.I. Is Learning From Humans. Many Humans.:

Labor-intensive data labeling powers AI. Web annotation technology is a key enabler for that labeling. I’m sure there will digital sweatshops running the software I help build. For what it’s worth, I’m doing my best to ensure there will be opportunities to elevate that work, empowering the humans in the loop to be decision-makers and exception-handlers, not just data-entry clerks.

Meanwhile, here’s that 1993 story, nearly forgotten by the web but salvaged from the NetNews group soc.culture.indian.

Small-systems thinking makes India a strategic software partner

by Jon Udell

BYTE, 09/01/1993

Recently, I saw a demonstration of a new Motif-based programmer’s tool called Sextant. It’s a source code analyzer that converts C programs into labeled graphs that you navigate interactively. The demonstration was impressive. What made it unique for me was that it took place in the offices of Softek, a software company in New Delhi, India.

It’s well known that Indian engineering talent pervades every level of the microcomputer industry. But what’s happening in India? On a recent tour of the country, I visited several high-tech companies and discovered that India is evolving rapidly from an exporter of computer engineering talent into an exporter of computer products and services. Software exports, in particular, dominate the agenda. A 1992 World Bank study of eight nations rated India as the most attractive nation for U.S., European, and Japanese companies seeking offshore software-development partners.

The World Bank study was conducted by Infotech Consulting (Parsippany, NJ). When the opportunity arose to visit India, I contacted Infotech’s president, Jack Epstein, for advice on planning my trip. He referred me to Pradeep Gupta, an entrepreneur in New Delhi who publishes two of India’s leading computer magazines, PC Quest and DataQuest. Gupta also runs market-research and conference businesses. He orchestrated a whirlwind week of visits to companies in New Delhi and Bombay and generously took time to acquaint me with the Indian high-tech scene.

A Nation of Small Systems

Even among Indians, there’s a tendency to attribute India’s emerging software prowess to the innate mathematical abilities of its people. “After all, we invented zero,” says Dewang Mehta, an internationally known computer graphics expert. He is also executive director of the National Association of Software and Service Companies (NASSCOM) in New Delhi. While this cultural stereotype may hold more than a grain of truth, it’s not the whole story. As NASSCOM’s 1992 report on the state of the Indian software industry notes, India has the world’s second-largest English-speaking technical work force. Consequently, Indian programmers are in tune with the international language of computing, as well as the language spoken in the U.S., the world’s largest market.

Furthermore, India’s data-communications infrastructure is rapidly modernizing. And the Indian government has begun an aggressive program of cutting taxes and lifting import restrictions for export-oriented Indian software businesses while simultaneously clearing the way for foreign companies to set up operations in India.

Other countries share many of these advantages, but India holds an ace. It is a nation of small systems. For U.S. and European companies that are right-sizing mainframe- and minicomputer-based information systems, the switch to PC-based client/server alternatives can be wrenching. Dumping the conceptual baggage of legacy systems isn’t a problem for India, however, because, in general, those systems simply don’t exist. “India’s mainframe era never happened,” says Gupta.

When Europe, Japan, and the U.S. were buying mainframes left and right, few Indian companies could afford their high prices, which were made even more costly by 150 percent import duties. Also, a government policy limiting foreign investors to a 40 percent equity stake in Indian manufacturing operations drove companies like IBM away, and the Indian mainframe industry never got off the ground.

What did develop was an indigenous microcomputer industry. In the early 1980s, Indian companies began to import components and to assemble and sell PC clones that ran DOS. This trend quickened in 1984, when the late Rajiv Gandhi, prime minister and an avid computer enthusiast, lifted licensing restrictions that had prevented clone makers from selling at full capacity.

In the latter half of the 1980s, a computerization initiative in the banking industry shifted the focus to Unix. Front offices would run DOS applications, but behind the scenes, a new breed of Indian-made PCs — Motorola- and Intel-based machines running Unix — would handle the processing chores. Unfortunately, that effort stalled when the banks ran afoul of the unions; even today, many of the Bank of India’s 50,000 branches aren’t linked electronically.

Nevertheless, the die was cast, and India entered the 1990s in possession of a special advantage. Indian programmers are not only well educated and English-speaking, but out of necessity they’re keenly focused on client/server or multiuser solutions for PCs running DOS (with NetWare) or Unix — just the kinds of solutions that U.S. and European companies are rushing to embrace. India finds itself uniquely positioned to help foreign partners right-size legacy applications.

The small-systems mind-set also guides India’s fledgling supercomputer industry. Denied permission by the U.S. government to import a Cray supercomputer, the Indian government’s Center for the Development of Advanced Computers built its own — very different — sort of supercomputer. Called PARAM, it gangs Inmos T800 transputers in parallel and can also harness Intel 860 processors for vector work. Related developments include a transputer-based neural-network engine intended to run process-control applications. The designers of this system impressed me with their clear grasp of the way in which inexpensive transputers can yield superior performance, scalability, modularity, and fault tolerance.

Software Products and Services

Many of the companies I visited produce comparable offerings for LAN or Unix environments. In the realm of packaged software, Oberoi Software in New Delhi sells a high-end hotel management application using Sybase 4.2 that runs on Hewlett-Packard, DEC, and Sun workstations. A low-end version uses Btrieve for DOS LANs. Softek offers 1-2-3, dBase, and WordStar work-alikes for DOS and Unix.

Shrink-wrapped products, however, aren’t India’s strong suit at the moment. PCs remain scarce and expensive commodities. According to DataQuest, fewer than 500,000 PCs can be found in this nation of 875 million people. To a U.S. software engineer, a $3000 PC might represent a month’s wages. An equivalently prosperous Indian professional would have to work a full year to pay for the same system. To put this in perspective, the average per capita wage in India is about $320, and the government caps the monthly salary of Indian corporate executives at around $1600 per month.

Software piracy is another vexing problem. “The competition for a 5000-rupee [approximately $165] Indian spreadsheet isn’t a 15,000-rupee imported copy of Lotus 1-2-3,” says Gupta, “but rather a zero-rupee pirated copy of Lotus 1-2-3.”

Painfully aware of the effect piracy has on the country’s international reputation as a software power, government and industry leaders have joined forces to combat it. The Department of Electronics (DoE), for example, has funded an anti-piracy campaign, and Lotus has a $69 amnesty program that enables users of illegal copies of 1-2-3 to come clean.

Reengineering Is a National Strength

The real action in Indian software isn’t in products. It’s in reengineering services. A typical project, for example, might involve re-creating an IBM AS/400-type application for a LAN or Unix environment. A few years ago, Indian programmers almost invariably would perform such work on location in the U.S. or Europe, a practice called “body shopping.” This was convenient for clients, but it wasn’t very beneficial to India because the tools and the knowledge spun off from reengineering projects tended to stay overseas.

More recently, the trend is to carry out such projects on Indian soil. Softek, for example, used a contract to build a law-office automation system for a Canadian firm as an opportunity to weld a number of its own products into a powerful, general-purpose client/server development toolkit. Softek engineers showed me how that toolkit supports single-source development of GUI software for DOS or Unix (in character mode) as well as Windows. They explained that client programs can connect to Softek’s own RDBMS (relational DBMS) or to servers from Gupta, Ingres, Oracle, or Sybase. That’s an impressive achievement matched by few companies anywhere in the world, and it’s one that should greatly enhance Softek’s appeal to foreign clients.

While reengineering often means right-sizing, that’s not always the case. For example, the National Indian Institution for Training, a New Delhi-based computer-training institute rapidly expanding into the realm of software products and services, has rewritten a well-known U.S. commercial word processor. Rigorous development techniques are the watchword at NIIT. “We have a passion for methodology,” says managing director Rajendra S. Pawar, whose company also distributes Excelerator, Intersolv’s CASE tool.

Other projects under way at NIIT include an X Window System interface builder, Mac and DOS tutorials to accompany the Streeter series of math textbooks (for McGraw-Hill), a simple but effective multimedia authoring tool called Imaginet, a word processor for special-needs users that exploits an NIIT-designed motion- and sound-sensitive input device, and an instructional video system.

Although services outweigh products for now, and the Indian trade press has complained that no indigenous software product has yet made a splash on the world scene, the situation could well change. Indian programmers are talented, and they’re up-to-date with database, GUI, network, and object-oriented technologies. These skills, along with wages 10 or more times less than U.S. programmers, make Indian programming a force to be reckoned with. Software development is a failure-prone endeavor; many products never see the light of day. But, as Tata Unisys (Bombay) assistant vice president Vijay Srirangan points out, “The cost of experimentation in India is low.” Of the many software experiments under way in India today, some will surely bear fruit.

A major obstacle blocking the path to commercial success is the lack of international marketing, but some help has been forthcoming. Under contract to the U.K.’s Developing Countries Trade Agency, the marketing firm Schofield Maguire (Cambridge, U.K.) is working to bring selected Indian software companies to the attention of European partners. “India does have a technological lead over other developing countries,” says managing partner Alison Maguire. “But to really capitalize on its software expertise, it must project a better image.”

Some companies have heard the message. For example, Ajay Madhok, a principal with AmSoft Systems (New Delhi), parlayed his firm’s expertise with computer graphics and digital video into a high-profile assignment at the 1992 Olympics. On a recent U.S. tour, he visited the National Association of Broadcasters show in Las Vegas. Then he flew to Atlanta for Comdex. While there, he bid for a video production job at the 1996 Olympics.

Incentives for Exporters

According to NASSCOM, in 1987, more than 90 percent of the Indian software industry’s $52 million in earnings came from “on-site services” (or body shopping). By 1991, on-site services accounted for a thinner 61 percent slice of a fatter $179 million pie. Reengineering services (and, to a lesser extent, packaged products) fueled this growth, with help from Indian and U.S. government policies.

On the U.S. side, visa restrictions have made it harder to import Indian software labor. India, meanwhile, has developed a range of incentives to stimulate the software and electronics industries. Government-sponsored technology parks in Noida (near New Delhi), Pune (near Bombay), Bangalore, Hyderabad, and several other locations support export-oriented software development. Companies that locate in these parks share common computing and telecommunications facilities (including leased-line access to satellite links), and they can import duty-free the equipment they need for software development.

The Indian government has established export processing zones in which foreign companies can set up subsidiaries that enjoy similar advantages and receive a five-year tax exemption. Outside these protected areas, companies can get comparable tax and licensing benefits by declaring themselves fully export-oriented.

Finally, the government is working to establish a number of hardware technology parks to complement the initiative in software. “We want to create many Hong Kongs and Singapores,” says N. Vittal, Secretary of the DoE and a tireless reformer of bureaucracy, alluding to the economic powerhouses of the Pacific Rim.

The Indian high-tech entrepreneurs I met all agreed that Vittal’s tenacious slashing of government red tape has blazed the trail they now follow. How serious is the problem of government red tape? When the government recently approved a joint-venture license application in four days, the action made headlines in both the general and trade press. Such matters more typically take months to grind their way through the Indian bureaucracy.

The evolution of India’s telecommunications infrastructure shows that progress has been dramatic, though uneven. In a country where only 5 percent of the homes have telephone service, high-tech companies increasingly rely on leased lines, packet-switched data networks, and satellite links. The DoE works with the Department of Telecommunication (DoT) to ensure that software export businesses get priority access to high-bandwidth services.

But the slow pace of progress at the DoT remains a major frustration. For example, faxing can be problematic in India, because the DoT expects you to apply for permission to transmit data. And despite widespread Unix literacy, only a few of the dozens of business cards I received during my tour carried Internet addresses. Why? DoT regulations have retarded what would have been the natural evolution of Unix networking in India. I did send mail home using ERNET, the educational resource network headquartered in the DoE building in New Delhi that links universities throughout the country. Unfortunately, ERNET isn’t available to India’s high-tech businesses.

Vittal recognizes the critical need to modernize India’s telecommunications. Given the scarcity of an existing telecommunications infrastructure, he boldly suggests that for many scattered population centers, the solution may be to completely pass over long-haul copper and vault directly into the satellite era. In the meantime, India remains in this area, as in so many others, a land of extreme contrasts. While most people lack basic telephone service, workers in strategic high-tech industries now take global voice and data services for granted.

Powerful Partners

When Kamal K. Singh, chairman and managing director of Rolta India, picks up his phone, Rolta’s U.S. partner, Intergraph, is just three digits away. A 64-Kbps leased line carries voice and data traffic from Rolta’s offices, located in the Santacruz Electronics Export Processing Zone (SEEPZ) near Bombay, to an earth station in the city’s center. Thence, such traffic travels via satellite and T1 lines in the U.S. to Intergraph’s offices in Huntsville, Alabama.

Rolta builds Intel- and RISC-based Intergraph workstations for sale in India; I saw employees doing everything from surface-mount to over-the-network software installation. At the same time, Rolta does facilities mapping for a U.S. telephone company through its subsidiary in Huntsville. Every night, scanned maps flow through the satellite link to Bombay. Operators running 386-based RoltaStations retrieve the maps from a Unix server, digitize them using Intergraph’s MicroStation CAD software, and relay the converted files back to Huntsville.

Many Indian companies have partnerships with U.S. firms. India’s top computer company, HCL, joined forces with Hewlett-Packard to form HCL-HP. HCL’s roots were in multiprocessor Unix. “Our fine-grained multiprocessing implementation of Unix System V has been used since 1988 by companies such as Pyramid and NCR,” says director Arjun Malhotra.

HCL’s joint venture enables it to build and sell HP workstations and PCs in India. “People appreciate HP quality,” says marketing chief Ajai Chowdhry. But since Vectra PCs are premium products in the price-sensitive Indian market, HCL-HP also plans to leverage its newly acquired HP design and manufacturing technology to build indigenous PCs that deliver “good value for money,” according to Malhotra.

Pertech Computers, a system maker in New Delhi, recently struck a $50 million deal to supply Dell Computer with 240,000 motherboards. Currently, trade regulations generally prohibit the import of certain items, such as finished PCs. However, exporters can use up to 25 percent of the foreign exchange they earn to import and sell such items. Pertech director Bikram Dasgupta plans to use his “forex” money to buy Dell systems for resale in India and to buy surface-mount equipment so that the company can build work-alikes.

IBM returned to India last year, after leaving in 1978, to join forces with the Tatas, a family of Indian industrialists. The joint venture, Tata Information Systems, will manufacture PS/2 and PS/VP systems and develop software exports.

Citicorp Overseas Software, a Citicorp subsidiary, typifies a growing trend to locate software-development units in India. “Our charter is first and foremost to meet Citicorp’s internal requirements,” says CEO S. Viswanathan, “but we are a profit center and can market our services and products.” On a tour of its SEEPZ facility in Bombay, I saw MVS, Unix, VMS, and Windows programmers at work on a variety of projects. In addition to reengineering work for Citicorp and other clients, the company markets banking products called Finware and MicroBanker.

ITC (Bangalore) supplements its Oracle, Ingres, and AS/400 consulting work by selling the full range of Lotus products. “Because we have the rights to manufacture Lotus software locally,” says vice president Shyamal Desai, “1-2-3 release 2.4 was available here within a week of its U.S. release.” Other distributors of foreign software include Onward Computer Technologies in Bombay (NetWare) and Bombay-based Mastek (Ingres).

India’s ambitious goal is to quadruple software exports from $225 million in 1992 to $1 billion in 1996. To achieve that, everything will have to fall into place. It would be a just reward. India gave much to the international microcomputer industry in the 1980s. In the 1990s, the industry just might return the favor.

“It’s not your fault, mom.”

I just found this never-published 2007 indictment of web commerce, and realized that if mom were still here 12 years later I could probably write the same thing today. There hasn’t been much much progress on smoother interaction for the impaired, and I don’t see modern web software development on track to get us there. Maybe it will require a (hopefully non-surgical) high-bandwidth brain/computer interface. Maybe Doc Searls’ universal shopping cart. Maybe both.

November 3, 2007

Tonight, while visiting my parents, I spent an hour helping my mom buy tickets to two Metropolitan Opera simulcasts. It went horribly wrong in six different ways. Here is the tale of woe.

A friend had directed her to Fandango with instructions to “Type ‘Metropolitan Opera’ in the upper-right search box.” Already my mom’s in trouble. Which upper-right search box? There’s one in the upper-right corner of the browser, and another in the upper-right corner of the Fandango page. She doesn’t distinguish between the two.

I steer her to the Fandango search box, she tries to type in ‘Metropolitan Opera’, and fails several times. Why? She’s unclear about clicking to set focus on the search box. And when she does finally aim for it, she misses. At age 86, arthritis and macular degeneration conspire against her.

I help her to focus on the search box, she types in ‘Metropolitan Opera’, and lands on a search results page. The title of the first show she wants to buy is luckily on the first page of a three-page result set, but it’s below the fold. She needs to scroll down to find it, but isn’t sure how.

I steer her to the show she wants, she clicks the title, and lands on another page where the interaction is again below the fold. Now she realizes she’s got to navigate within every page to the active region. But weak vision and poor manual dexterity make that a challenge.

We reach the page for the April 26th show. Except, not quite. Before she can choose a time for the show — unnecessarily, since there’s only one time — she has to reselect the date by clicking a link labeled ‘Saturday April 26th’. Unfortunately the link text’s color varies so subtly from the regular text’s color that she can’t see the difference, and doesn’t realize it is a link.

I steer her to the link, and she clicks to reveal the show time: 1:30. Again it’s unclear to her that this is a link she must follow.

I steer her to the 1:30 link, and finally we reach the purchase page. Turns out she already has an account with Fandango, which my sister must have helped her create some time ago. So mom just needs to sign in and…

You can see this coming from a mile away. She’s forgotten which of her usual passwords she used at this site. After a couple of failures, I steer her to the ‘Forgot password’ link, and we do the email-checking dance.

The email comes through, we recover the password, and proceed to checkout. The site remembers her billing address and credit card info. I’m seeing daylight at the end of the tunnel.

Except it’s not daylight, it’s an oncoming train.

Recall that we’re trying to buy tickets for two different shows. So now it’s time to go back and add the second one to the shopping cart. Unfortunately Fandango doesn’t seem to have a shopping cart. Each time through you can only buy one set of tickets.

I steer her back to the partially-completed first transaction. We buy those tickets and print the confirmation page.

Then we enter the purchase loop for the second time and…this one is not like the other one. This time through, it asks for the card’s 3-digit security code. She enters it correctly, but the transaction fails because something has triggered the credit card company’s fraud detector. Probably the two separate-but-identical charges in rapid succession.

We call the credit card company, its automated system wants her to speak or enter her social security number. She tries speaking the number, the speech recognizer gets it wrong, but mom can’t hear what it says back to her. In addition to macular degeneration and arthritis, she’s lost a lot of her hearing in the last few years. So she tries typing the social security number on the phone’s keypad, and fails.

I grab the phone and repeat ‘agent’ and ‘operator’ until somebody shows up on the line. He does the authentication dance with her (maiden name, husband’s date of birth, etc.), then I get back on the line and explain the problem. The following dialogue ensues:

Agent: “Whoa, this is going to be hard, we’re having a problem and I can’t access her account right now. Do you want to try later?”

Me: “Well, I’m here with my 86-year-old mom, and we’ve invested a whole lot of effort in getting to this point, is there any way to hang onto the context?”

He sympathizes, and connects me to another, more powerful agent. She overrides the refusal, authorizes the $63, and invites me to try again.

Now the site reports a different error: a mismatch in either the billing zip code, or the credit card’s 3-digit security code. To Fandango it looks like the transaction failed. To the credit card company it looks like it succeeded. What now?

The agent is pretty sure that if the transaction failed from Fandango’s perspective, it’ll come out in the wash. But we’re both puzzled. I’m certain the security code is correct. And all other account info must be correct, right? How else how could we have successfully bought the first set of tickets?

But just to double-check, I visit mom’s account page on Fandango. The billing address zip code is indeed correct. So is everything else, except…wait…the credit card’s expiration date doesn’t match. The account page says 11/2007, the physical card says 11/2010.

Turns out the credit card company recently refreshed mom’s card. This is weird and disturbing because, if the successful transaction and the failed transaction were both using the same wrong expiration date, they both should have failed.

Sigh. I change the expiration date, and finally we buy the second set of tickets. It’s taken an hour. My mom observes that if Fandango accepted orders over the phone, we’d have been done about 55 minutes ago. And she asks:

“How could I possibly have done that on my own?”

I know all the right answers. Better web interaction design. Better assistive technologies for the vision-, hearing-, and dexterity-impaired. Better service integration between merchants and credit card companies. Better management of digital identity. Someday it’ll all come together in a way that would enable my mom to do this for herself. But today isn’t that day.

Her only rational strategy was to do just what she did, namely recruit me. For which she apologizes.

All I can say, in the end, is: “Mom, it’s not your fault.”

Walking the Kortum Trail

On my first trip to Sonoma County I flew to SFO and drove north through San Francisco, across the Golden Gate Bridge, and through Marin County on the 101 freeway.

The next time I drove highway 1 up the coast and was stunned by the contrast. The Marin headlands are twisty and challenging, but then the road opens out to a series of ocean and grassland vistas inhabited mainly, it seems, by a few lucky cattle. How, I wondered, could such pristine coastline exist so near the megalopolis?

One especially magical stretch runs north from Bodega Bay, on the Marin/Sonoma county line, to Jenner at the mouth of the Russian River. If you park at Wrights Beach you can walk the last four miles to the river, along dramatic coastal bluffs, on a trail named for Bill and Lucy Kortum. When Bill died in 2015 the Press Democrat described him as a man “who spent most of his life fighting to rein in urban sprawl and protect public access to the coast” and “saved the landscape of Sonoma County.”

Recently, for the first time, I hiked the Kortum Trail end-to-end both ways. Here’s the view from Peaked Hill:

For that view, and for so much else that we love about Sonoma County, we owe thanks to Bill Kortum. It began in the early 1960s when his older brother Karl led the fight to stop the construction of a nuclear power plant on Bodega Head. Although I’ve changed my mind about the need for nuclear power, siting one on the San Andreas fault was a crazy plan. Thankfully that never happened.

We’re frequent visitors to Bodega Head, and we’d heard about the “Hole in the Head” — the excavated pit left behind when PG&E abandoned the project — but until recently we weren’t sure quite where it was. Turns out we’d driven right past it every time we made the hairpin turn on Westshore Road at Campbell Cove. What would have been a nuclear plant is now a tranquil pond enjoyed by waterfowl.

Many of the things we love about Sonoma County can be traced to the activism that began there, and has continued for fifty years, led or influenced by Bill Kortum.

The planned community at Sea Ranch would have blocked public access to 13 miles of coastline. That didn’t happen. Now there are six access trails.

The entire California coastline would look very different were it not for the oversight of the California Coastal Commission, established by voters in 1972 partly in response to the Sea Ranch controversy.

The Sonoma County landscape would look very different were it not for urban growth boundaries that restrict sprawl and encourage densification.

The shameful six-decade-long lack of a passenger rail system in the North Bay would have continued.

The 1200-mile California Coastal Trail would not be more than halfway complete.

I was a bit surprised to find no Wikipedia page for Bill Kortum, so I’ve created a stub, about which Wikipedia says: “Review waiting, please be patient. This may take 8 weeks or more.”

As a relative newcomer to Sonoma County I feel ill-equipped to write more of this history. Perhaps others who know more will add to that stub. Here are some of the best sources I’ve found:

– John Crevelli’s Fifty Year History of Environmental Activism in Sonoma County.

– Gaye LeBaron’s 2006 interview with Bill Kortum.

– The Press Democrat’s obituary.

You were nearing the end of your life when we arrived here, Bill. I never had the chance to meet you, or to thank you for having the vision to preserve so much that is special about this place, along with the will and political savvy to make it happen. We are deeply grateful.

Why Maine embraces immigration

An excellent choice for 4th-of-July podcast listening would be Maine, Africa on Radio Open Source. The intro:

We went to Portland, Maine, this week to meet newcomers from Central Africa, Angolans and Congolese asking for U.S. asylum. Fox News hit the panic button two weeks ago: their line was that Maine is being overrun, inundated by African migrants. On a long day in Portland, however, we found nobody sounding scared.

I’ve noticed the strong African influence in Portland when visiting, and wondered: “Why is that?” The answer is that Maine’s population is stable but aging. Immigrants are needed, and the state welcomes them.

A couple of moments in this podcast stopped me in my tracks.

With mayor Ethan Strimling:

“You know what, we don’t consider you somebody else’s problem, we consider you our opportunity.”

And with a woman interviewed on the street:

Chris Lydon: “How did Portland get to be so welcoming?”

Interviewee: “We’re Americans.”

Not everyone’s on board. There’s an anti-immigrant minority, but it is a minority. Most of the people in Portland, it seems, look backward to their own immigrant roots and forward to a future that embraces an influx of world cultures.

On this Independence Day I choose to believe that a national majority feels the same way, and will prevail.

The Woodard projection

In a memorable episode of The West Wing, visitors from the Cartographers for Social Justice upend CJ’s and Josh’s worldviews.

Cartographer: “The Peters projection.”

CJ: “What the hell is that?”

Cartographer: “It’s where you’ve been living this whole time.”

I’m having the same reaction to Colin Woodard’s 2011 book American Nations. He sees North America as three federations of nations. The federation we call the United States comprises nations he calls Yankeedom, New Netherland, The Midlands, Tidewater, Greater Appalachia, The Deep South, El Norte, The Far West, and The Left Coast.

Here’s his definition of a nation:

A nation is a group of people who share — or believe they share — a common culture, ethnic origin, language, historical experience, artifacts, and symbols.”

Worldwide some nations are stateless, some align with state borders, and some cut across state borders. North America’s eleven nations are, in his view, stateless, cutting across state boundaries in ways I find disorienting but enlightening.

On his map (from a 2013 Tufts alumni magazine article now readable only at the Internet Archive) I have marked the US places where I’ve lived.

Until now, if you asked me where I’ve lived, I’d have said East Coast and Midwest and recently California. According to the Woodard projection I have lived in four nations: Yankeedom, The Midlands, Tidewater, and The Left Coast. It wasn’t easy to locate my homes on his map. They all occupy a narrow band of latitude. On the East Coast, that band touches three of Woodard’s nations. In two of those, Yankeedom and The Midlands, I lived near the cradles of nations that spread far north and west.

I’m from near Philadelphia, in The Midlands, “founded by English Quakers, who welcomed people of many nations and creeds to their utopian colonies on the shores of Delaware Bay.” That resonates. I grew up in a place called Plymouth Meeting and went to Quaker kindergarten there. It would never have occurred to me that Philadelphia is culturally closer to places as far west as Nebraska, and as far north as the province of Ontario, than to Baltimore or Boston. Likewise I never thought of Ann Arbor, where I called myself a midwesterner, as part of a culture that flowed west from Boston. Or that Baltimore sits at the intersection of three nations.

These eleven nations have been hiding in plain sight throughout our history. You see them outlined on linguists’ dialect maps, cultural anthropologists’ maps of material culture regions, cultural geographers’ maps of religious regions, campaign strategists’ maps of political geography, and historians’ maps of the pattern of settlement across the continent.

Two of the eleven nations, Yankeedom and The Deep South, have been “locked in nearly perpetual combat for control of the federal government since the moment such a thing existed,” Woodard says.

The analysis, which builds on prior art that he cites, may be a helpful way to contextualize the 2016 US election.

“The Woodard projection.”

“What the hell is that?”

“It’s where you’ve been living this whole time.”

Don’t just Google it! First, let’s talk!

Asking questions in conversation has become problematic. For example, try saying this out loud: “I wonder when Martin Luther King was born?” If you ask that online, a likely response is: “Just Google it!” Maybe with a snarky link: was martin luther king born?

Asking in conversation is frowned upon too. Why ask us when you could ask Siri or Alexa to ask Google?

There’s a kind of shaming going on here. We are augmented with superpowers, people imply, you’re an idiot to not use them.

Equally problematic is the way this attitude shuts down conversation. Sure, I can look up MLK’s birthday. Or you can. But what if our phones are dead? Now we need to rely on our own stored knowledge and, more importantly, on our collective pool of stored knowledge.

I think MLK was shot in 1968. I’d have been twelve. Does that sound right? Yeah, we were in the new house, it was sixth grade. And I know he died young, maybe 42? So I’ll guess 1968 – 42 = 1926.

Were you around then? If so, how do you remember MLK’s assassination? If not, what do you know about the event and its context?

As you can see in the snarky screencast, I’m wrong, MLK died even younger than I thought. You might know that. If we put our heads together, we might get the right answer.

Asking about something that can easily be looked up shouldn’t stop a conversation, it should start one. Of course we can look up the answer. But let’s dwell for a minute on what we know, or think we know. Let’s exercise our own powers of recall and synthesis.

Like all forms of exercise, this one can be hard to motivate, even though it’s almost always rewarding. Given a choice between an escalator and the stairs, I have to push myself to prefer the stairs. In low-stakes wayfinding I have to push myself to test my own sense of direction. When tuning my guitar I have to push myself to hear the tones before I check them. I am grateful to be augmented by escalators, GPS wayfinders, and tuners. But I want to use these powers in ways that complement my own. Finding the right balance is a struggle.

Why bother? Consider MLK. In The unaugmented mind I quoted from MLK confidante Clarence Jones’ Behind the Dream:

What amazed me was that there was absolutely no reference material for Martin to draw upon. There he was [in the Birmingham jail] pulling quote after quote from thin air. The Bible, yes, as might be expected from a Baptist minister, but also British prime minister William Gladstone, Mahatma Gandhi, William Shakespeare, and St. Augustine.


The “Letter from a Birmingham Jail” showed his recall for the written material of others; his gruelling schedule of speeches illuminated his ability to do the same for his own words. Martin could remember exact phrases from several of his unrelated speeches and discover a new way of linking them together as if they were all parts of a singular ever-evolving speech. And he could do it on the fly.

Jones suggests that MLK’s power flowed in part from his ability to reconfigure a well-stocked working memory. We owe it to ourselves, and to one another, to nurture that power. Of course we can look stuff up. But let’s not always go there first. Let’s take a minute to exercise our working memories and have a conversation about them, then consult the oracle. I’ll bet that in many cases that conversation will turn out to be the most valuable part of the experience.

My new library superpower

The recommendations that matter to me — for books to read, movies to watch, products to buy, places to visit — almost never come from algorithms. Instead they come from friends, family, acquaintances, and now also, in the case of books, from fellow library patrons whom I’ve never met.

This last source is a recent discovery. Here’s how it works. When I enter the library, I walk past shelves of new books and staff picks. These are sometimes appealing, but there’s a much better source of recommendations to be found at the back of the library. There, on shelves and carts, are the recently-returned books ready to be reshelved. I always find interesting titles among them. Somehow I had never noticed that you can scan these titles, and check out the ones you want before they ever get reshelved. Did it always work this way at our library? At libraries in other places I’ve lived? If so I am woefully late to the party, but for me, at least, this has become a new superpower.

I reckon that most books in the library rarely if ever leave the shelves. The recently-returned section is a filter that selects for books that patrons found interesting enough to check out, read (or maybe not), and return. That filter has no bias with respect to newness or appeal to library staff. And it’s not being manipulated by any algorithm. It’s a pure representation of what our library’s patrons collectively find interesting.

The library’s obvious advantage over the bookstore is the zero price tag. But there’s a subtler advantage. In the bookstore, you can’t peruse an anonymized cache of recently-bought books. Online you can of course tap into the global zeitgeist but there you’re usually being manipulated by algorithms. LibraryThing doesn’t do that, and in principle it’s my kind of place online, but in practice I’ve never succeeded in making it a habit. I think that’s because I really like scanning titles that I can just take off a shelf and read for free. Not even drone delivery can shorten the distance between noticing and reading to seconds. Ebooks can, of course. But that means another screen, I already spend too much time looking at screens, the print medium provides a welcome alternative.

This method has been so effective that I’ve been (guiltily) a bit reluctant to describe it. After all, if everyone realized it’s better to bypass the new-book and staff-pick displays and head straight for the recently-returned area, the pickings would be slimmer there. But I’m nevertheless compelled to disclose the secret. Since few if any of my library’s patrons will ever read this post, I don’t think I’m diluting my new superpower by explaining it here. And who knows, maybe I’ll meet some interesting people as I exercise it.


Designing for least knowledge

In a post on the company blog I announced that it’s now possible to use the Hypothesis extension in the Brave browser. That’s great news for Hypothesis. It’s awkward, after all, to be an open-source non-profit software company whose product is “best viewed in” Chrome. A Firefox/Hypothesis extension has been in the works for quite a while, but the devil’s in the details. Although Firefox supports WebExtensions, our extension has to be ported to Firefox, it isn’t plug-and-play. Thanks to Brave’s new ability to install unmodified Chrome extensions directly from the Chrome web store, Hypothesis can now plug into a powerful alternative browser. That’s a huge win for Hypothesis, and also for me personally since I’m no longer tethered to Chrome in my daily work.

Another powerful alternative is forthcoming. Microsoft’s successor to Edge will (like Brave) be built on Chromium, the open-source engine of Google’s Chrome. The demise of EdgeHTML represents a loss of genetic diversity, and that’s unfortunate. Still, it’s a huge investment to maintain an independent implementation of a browser engine, and I think Microsoft made the right pragmatic call. I’m now hoping that the Chromium ecosystem will support speciation at a higher level in the stack. Ever since my first experiments with Greasemonkey I’ve been wowed by what browser extensions can do, and eager for standardization to unlock that potential. It feels like we may finally be getting there.

Brave’s support for Chrome extensions matters to me because I work on a Chrome extension. But Brave’s approach to privacy matters to me for deeper reasons. In a 2003 InfoWorld article I discussed Peter Wayner’s seminal book Translucent Databases, which explores ways to build information systems that work without requiring the operators to have full access to users’ data. The recipes in the book point to a design principle of least knowledge.

Surveillance capitalism knows way too much about us. Is that a necessary tradeoff for the powerful conveniences it delivers? It’s easy to assume so, but we haven’t really tried serious alternatives yet. That’s why this tweet made my day. “We ourselves do not know user identities or how donations might link via user,” wrote Brendan Eich, Brave’s founder. “We don’t want to know.”

We don’t want to know!

That’s the principle of least knowledge in action. Brave is deploying it in service of a mission to detoxify the relationship between web content and advertising. Will the proposed mechanisms work? We’ll see. If you’re curious, I recommend Brendan Eich’s interview with Eric Knorr. The first half of the interview is a deep dive into JavaScript, the second half unpacks Brave’s business model. However that model turns out, I’m grateful to see a real test of the principle. We need examples of publishing and social media services that succeed not as adversaries who monetize our data but rather as providers who deliver convenience at a reasonable price we’re happy to pay.

My hunch is that we’ll find ways to build profitable least-knowledge services once we start to really try. Successful examples will provide a carrot, but there’s also a stick. Surveillance data is toxic stuff, risky to monetize because it always spills. It’s a liability that regulators — and perhaps also insurers — will increasingly make explicit.

Don’t be evil? How about can’t be evil? That’s a design principle worth exploring.

Highlighting passages doesn’t aid my memory, but speaking them does

When I was in college, taking notes on textbooks and course readings, I often copied key passages into a notebook. There weren’t computers then, so like a medieval scribe I wrote out my selections longhand. Sometimes I added my own notes, sometimes not, but I never highlighted, even in books that I owned. Writing out the selections was a way to perform the work I was reading, record selections in my memory, and gain deeper access to the mind of the author.

Now we have computers, and the annotation software I help build at Hypothesis is ideal for personal note-taking. Close to half of all Hypothesis annotations are private notes, so clearly lots of people use it that way. For me, though, web annotation isn’t a private activity. I bookmark and tag web resources that I want to keep track of, and collaborate with others on document review, but I don’t use web annotation to enhance my private reading.

To be sure, I mostly read books and magazines in print. It’s a welcome alternative to the screens that otherwise dominate my life. But even when my private reading happens online, I don’t find myself using our annotation tool the way so many others do.

So, what’s a good way to mark and remember a passage in a book if you don’t want to highlight it, or in the case of a library book, can’t highlight it? I thought about the scribing I used to do in college, and realized there’s now another way to do that. Recently, when I read a passage in a book or magazine that I want to remember and contemplate, I’ve been dictating it into a note-taking app on my phone.

I’ve followed the evolution of speech-to-text technology with great interest over the years. When I reviewed Dragon NaturallySpeaking, I did what every reviewer does. I tried to use the tool to dictate my review, and got mixed results. Over time the tech improved but I haven’t yet adopted dictation for normal work. At some point I decided to forget about dictation software until it became something that civilians who weren’t early-adopter tech journos used in real life.

One day, when I received some odd text messages from Luann, I realized that time had arrived. She’d found the dictation feature on her phone. It wasn’t working perfectly, and the glitches were amusing, but she was using it in an easy and natural way, and the results were good enough.

I still don’t dictate to my computer. This essay is coming to you by way of a keyboard. But I dictate to my phone a lot, mostly for text messages. The experience keeps improving, and now this new practice — voicing passages that I read in books, in order to capture and remember them — seems to be taking hold.

I’m reminded of a segment in a talk given by Robert “R0ml” Lefkowitz at the 2004 Open Source Conference, entitled The Semasiology of Open Source (part 2), the second in a series structured as thesis (part 1), antithesis (part 2), and synthesis (part 3). ITConversation aptly described this luminous series of talks as “an intellectual joy-ride”; I’m going to revisit the whole thing on a hike later today.

Meanwhile, here’s a transcription of the segment I’m recalling. It appears during a review of the history of literacy. At this point we have arrived at 600 AD.

To be a reader was not to be the receiver of information, it was to be the transmitter of the information, because it was not possible to read silently. So things that were written were written as memory aids to the speaker. And the speaker would say the words to the listener. To read was to speak, and those were synonyms … The writing just lies there, whereas the speaking lifts it off the page. The writing is just there, but the speaking is what elevates the listener.

Had I merely read that passage I’m certain I would not remember it 14 years later. Hearing R0ml speak the words made an indelible impression. (Seeing him speak the words, of course, made it even more indelible.)

Silent reading, once thought impossible, had to be invented. But just because we can read silently doesn’t mean we always should, as everyone who’s read aloud to a young child, or to a vision-impaired elder, knows. It’s delightful that voice recognition affords new ways to benefit from the ancient practice of reading aloud.

A small blog neighborhood hiding in plain sight

For as long as I can remember, I’ve tweeted every blog post. As an experiment, I didn’t do that with this one. As a result, WordPress tells me that relatively few people have read it. But I’m not monetizing pageview counters here, why do I care? The interaction with those who did read that item was pleasant.

So, the experiment continues. Some items here, like this one, can enjoy a more intimate space than others, simply by not announcing themselves on Twitter. A small blog neighborhood hiding in plain sight.

Where’s my Net dashboard?

Yesterday Luann was reading a colleague’s blog and noticed a bug. When she clicked the Subscribe link, the browser loaded a page of what looked like computer code. She asked, quite reasonably: “What’s wrong? Who do I report this to?”

That page of code is an RSS feed. It works the same way as the one on her own blog. The behavior isn’t a bug, it’s a lost tradition. Luann has been an active blogger for many years, and once used an RSS reader, but for her and so many others, the idea of a common reader for the web has faded.

There was a time when most of the sources I cared about flowed into such a reader: mainstream news, a vibrant and growing blogosphere, podcasts, status updates, standing search queries, you name it. The unread item count could get daunting, but I was able to effectively follow a wide assortment of information flows in what I came to think of as my Net dashboard.

Where’s my next dashboard? I imagine a next-gen reader that brings me the open web and my social circles in a way that helps me attend to and manage all the flow. There are apps for that, a nice example being FlowReader, which has been around since 2013. I try these things hopefully but so far none has stuck.

Information overload, once called infoglut, remains a challenge. We’re all flooded with more channels than we can handle, more conversations happening in more places than we can keep track of.

Fear of missing out (FOMO) is the flip side of infoglut. We expect that we should be able to sanely monitor more than we actually can.

The first-gen reader didn’t solve infoglut/FOMO, nothing could, but for a while, for me, it was better than the alternative, which was (and now is again) email. Of course that was me, a tech journalist who participated in, researched, and wrote about topics in Net technology and culture — including RSS, which animated the dashboard I used to keep track of everything else. It was, however, a workflow that researchers and analysts in other fields will recognize.

Were I were doing the same kind of work today, I’d cobble together the same kind of dashboard, while grumbling about the poorer experience now available. Instead my professional information diet is narrower and deeper than when analytical writing for commercial audiences was my work. My personal information diet, meanwhile, remains as diverse as everyone’s.

So I’m not sure that a next-gen reader can solve the same problems that my first-gen reader did, in the same ways. Still, I can’t help but envision a dashboard that subscribes to, and manages notifications from, all my sources. It seems wrong that the closest thing to that, once more, is email. Plugging the social silos into a common reader seems like the obvious thing. But if that were effective, we’d all be using FlowReader or something like it.

Why don’t we? Obviously the silos can refuse to cooperate, as FlowReader noted when announcing the demise of its Facebook integration:

These changes were made [by Facebook] to give users more control over their own data, which we support. It’s a great thing for users! However, it also means that Facebook data is no longer going to be easy to share between applications.

You know what would be a really great thing for users, though? A common reader that makes it easy to keep track of friends and family and coworkers along with news and all kinds of personal and professional information sources.

“What’s wrong?”

It’s not just that the silos can shut down their feeds. It’s that we allowed ourselves to get herded into them in the first place. For a while, quite a few people got comfortable with the notion of publishing and subscribing to diverse feeds in a common way, using systems that put them in charge of outflow and inflow. In one form or another that’s still the right model. Sometimes we forget things and have to relearn them. This is one of those things.

“Who do I report this to?”


The reengineering of three California lakes

If you drive up the the eastern side of California, you’ll encounter three ancient lakebeds transformed by human engineering during the last century. The sequence goes like this: Salton Sea, Owens Valley, Mono Lake. We visited all three on a recent road trip. Since returning I’ve learned that their stories continue to unfold.

The Salton Sea

The Salton Sea, almost twice the size of Lake Tahoe, was created accidentally in 1905 when an irrigation project went out of control and, for a year and a half, sucked the Colorado river into what had been a dry lakebed. As a recent immigrant to California I can confirm that many natives don’t know much, if anything, about this place. A 2004 documentary narrated by John Waters, Plagues & Pleasures on the Salton Sea, traces its arc through a boom era of sport fishing and real estate speculation to what is now a living ghost town. For decades people have talked about saving the Salton Sea. That may once have meant restoring California’s “lost Riviera” but nowadays it’s all about mitigating an environmental disaster.

The Salton Sink is a low-lying basin that has flooded and dried many times over hundreds of thousands of years. What makes the current drying phase different is that the only inflow now is agricultural runoff with high concentrations of pesticides. As the lake evaporates, toxic dust goes windborne, threatening not only the Salton Sea communities, but also nearby Mexicali, Mexico, a metropolitan area with a million people. Meanwhile the lake’s increasing salinity is ruining the habitat of millions of migratory birds. These looming health and environmental crises motivated California to allocate $200 million as part of a successful June 2018 ballot initiative (Prop. 68) to stabilize the Salton Sea. (Another $200 million won’t be forthcoming because it was part of a failed follow-on ballot initiative in November 2018, Prop. 3.)

In an effort to buy time, a 2003 agreement to transfer water from the Imperial Valley to San Diego required the release of “mitigation water” to the Salton Sea. That ended in 2017 with no clear roadmap in place. What would it mean to stabilize the Salton Sea? A dwindling Colorado River won’t be the answer. It may be possible to import seawater from the Pacific, or the Gulf of California, to create a “smaller but sustainable” future that balances the needs of the region’s people and wildlife in a context of growing crises of drought and fire. Dry methods of dust suppression might also help. But all methods will require major engineering of one kind or another. The cost of that 1905 accident continues to grow.

The Owens Valley

Further north lies Owens Lake, mostly drained since 1913 when William Mulholland began siphoning its water into the LA aqueduct. The movie Chinatown is a fictionalized account of the battle for that water. What remains is mostly a vast salt flat that’s patrolled by mining vehicles and was, until recently, the nation’s worst source of dust pollution. From highway 395 it still looks like the surface of some other planet. But mitigation began at the turn of this century, and by 2017 LA’s Department of Water and Power had spent nearly $2 billion on what a modern western explorer known as StrayngerRanger describes as “a patchwork of dust-smothering techniques, including gravel, flooded ponds and rows of planted vegetation that cover nearly 50 square miles — an area more than twice the size of Manhattan.”

A key innovation, according to the LA Times, “involves using tractors to turn moist lake bed clay into furrows and basketball-sized clods of dirt. The clods will bottle up the dust for years before breaking down, at which point the process will be repeated.” This dry method was a big success, the dust has settled, public trails opened in 2016, and as Robin Black’s photography shows, life is returning to what’s left of the lake.

The LA Times:

In what is now hailed as an astonishing environmental success, nature quickly responded. First to appear on the thin sheen of water tinged bright green, red and orange by algae and bacteria were brine flies. Then came masses of waterfowl and shorebirds that feed on the insects.

Robin Black:

This was never supposed to happen, and it’s a BIG DEAL. This, the habitat creation and management. This, the welcoming of the public to the lake. This is a huge victory for the groups in the Owens Valley who fought so tirelessly to make this happen.

We need to celebrate environmental successes and learn from them. And on this trip, the best one was yet to come.

Mono Lake

Mono Lake, near the town of Lee Vining (“the gateway to Yosemite”), was the next object of LA’s thirst. Its inflow was diverted to the aqueduct, and the lake almost went dry. Now it’s filling up again, thanks to the dedicated activists who formed the Mono Lake Committee 40 years ago, sued the LA Department of Water and Power, won back 3/4 of the inflow, and as chronicled in their video documentary The Mono Lake Story, have been stewarding the lake’s recovery ever since.

Today you can walk along the newly-restored Lee Vining Creek trail and feel the place coming alive again. There’s been cleanup and trail-building and planting but, for the most part, all that was needed was to let the water flow again.

How did LA compensate for the lost flow? This 30-second segment of the documentary answers the question. The agreement included funding that enabled LA to build a wastewater reclamation plant to make up the difference. It wasn’t a zero-sum game after all. There was a way to maintain a reliable water supply for LA and to restore Mono Lake. It just took people with the vision to see that possibility, and the will to make it real.

Renaming Hypothesis tags

Wherever social tagging is supported as an optional feature, its use obeys a power law. Some people use tags consistently, some sporadically, most never. This chart of Hypothesis usage illustrates the familiar long-tail distribution:

Those of us in the small minority of consistent taggers care a lot about the tag namespaces we’re creating. We tag in order to classify resources, we want to be able to classify them consistently, but we also want to morph our tag namespaces to reflect our own changing intuitions about how to classify and also to adapt to evolving social conventions.

Consistent tagging requires a way to make, use, and perhaps share a list of controlled tags, and that’s a topic for another post. Morphing tag namespaces to satisfy personal needs, or adapt to social conventions, requires the ability to rename tags, and that’s the focus of this post.

There’s nothing new about the underlying principles. When I first got into social tagging, with back in the day, I made a screencast to show how I was using’ tag renaming feature to reorganize my own classification system, and to evolve it in response to community tag usage. The screencast argues that social taggers form a language community in which metadata vocabularies can evolve in the same way natural languages do.

Over the years I’ve built tag renamers for other systems, and now I’ve made one for Hypothesis as shown in this 90-second demo. If you’re among the minority who want to manage your tags in this way, you’re welcome to use the tool, here’s the link. But please proceed with care. When you reorganize a tag namespace, it’s possible to wind up on the wrong end of the arrow of entropy!

Letters to Mr. Wilson’s Museum of Jurassic Technology

Dear Mr. Wilson,

Your Museum of Jurassic Technology (MJT) first came to my attention in 1995 when I read an excerpt from Lawrence Weschler’s Mr. Wilson’s Cabinet of Wonder in the New Yorker. Or anyway, that’s the origin story I’ve long remembered and told. As befits the reality-warping ethos of the MJT, it seems not to be true. I can find no record of such an article in the New Yorker’s archive. (Maybe I read it in Harpers?) What I can find, in the New York Times’ archive, is a review of the book that nicely captures that ethos:

Run by its eccentric proprietor out of a storefront in Culver City, Calif., the museum is clearly a modern-day version, as Mr. Weschler astutely points out, of the “wonder-cabinets” that sprang up in late Renaissance Europe, inspired by all the discoveries in the New World. David Wilson comes off as an amusingly Casaubonesque figure who, in his own little way, seeks to amass all the various kinds of knowledge in the world; and if his efforts seem random and arcane, they at any rate sound scientifically specific. Yet when Mr. Weschler begins to check out some of the information in the exhibits, we discover that much of it is made up or imagined or so elaborately embroidered as to cease to resemble any real-world facts.

The key to the pleasure of this book lies in that “much of,” for the point of David Wilson’s museum is that you can’t tell which parts are true and which invented. In fact, some of the unlikeliest items — the horned stink ants, for instance — turn out to be pretty much true. In the wake of its moment of climactic exposure, “Mr. Wilson’s Cabinet of Wonder” turns into an expedition in which Lawrence Weschler tracks down the overlaps, correspondences and occasionally tenuous connections between historical and scientific reality on the one hand and the Museum of Jurassic Technology on the other.

We’ve always wanted to visit the MJT, and finally made the pilgrimage a few weeks ago. Cruising down the California coast on a road trip, the museum was our only LA destination. The miserable traffic we fought all day, both entering and leaving Culver City, was a high price to pay for several hours of joy at the museum. But it was worth it.

At one point during our visit, I was inspecting a display case filled with a collection of antique lace, each accompanied by a small freestanding sign describing the item. On one of the signs, the text degrades hilariously. As I remember it (no doubt imperfectly) there were multiple modes of failure: bad kerning, faulty line spacing, character misencoding, dropouts, faded print. Did I miss any?

While I was laughing at the joke, I became aware of another presence in the dark room, a man with white hair who seemed to pause near me before wandering off to an adjoining dark room. I thought it might be you, but I was too intimidated to ask.

The other night, looking for videos that might help me figure out if that really was you, I found a recording of an event at the USC Fisher Museum of Art. In that conversation you note that laughter is a frequent response to the MJT:

“We don’t have a clue what people are laughing about,” you said, “but it’s really hard to argue with laughter.”

I was laughing at that bit of hilariously degraded signage. I hope that was you hovering behind me, and I hope that you were appreciating my appreciation of the joke. But maybe not. Maybe that’s going to be another questionable memory. Whatever the truth may be, thank you. We need the laughs now more than ever.

Yours sincerely,

Jon Udell

Searching across silos, circa 2018

The other day, while searching across various information silos — WordPress, Slack,, GitHub, Google Drive, Google Groups, Zendesk, Stack Overflow for Teams — I remembered the first time I simplified that chore. It was a fall day in 1996. I’d been thinking about software components since writing BYTE’s Componentware1 cover story in 1994, and about web software since launching the BYTE website in 1995. On that day in 1996 I put the two ideas together for the first time, and wrote up my findings in an installment of my monthly column entitled On-Line Componentware. (Yes, we really did write “On-Line” that way.)

The solution described in that column was motivated by McGraw-Hill’s need to unify search across BYTE and some other MgH pubs that were coming online. Alta Vista, then the dominant search engine, was indexing all the sites. It didn’t offer a way to search across them. But it did offer a site: query as Google and Bing do today, so you could run per-site queries. I saw it was possible to regard Alta Vista as a new kind of software component for a new kind of web-based application that called the search engine once per site and stitched the results into a single web page.

Over the years I’ve done a few of these metasearch apps. In the pre-API era that meant normalizing results from scraped web pages. In the API era it often means normalizing results fetched from per-site APIs. But that kind of normalization was overkill this time around, I just needed an easier way to search for words that might be on our blog, or in our docs, or in our GitHub repos, or in our Google storage. So I made a web page that accepts a search term, runs a bunch of site-specific searches, and opens each into a new tab.

This solution isn’t nearly as nice as some of my prior metasearchers. But it’s way easier than authenticating to APIs and merging their results, or using a bunch of different query syntaxes interactively. And it’s really helping me find stuff scattered across our silos.

But — there’s always a but — you’ll notice that Slack and Zendesk aren’t yet in the mix. All the other services make it possible to form a URL that includes the search term. That’s just basic web thinking. A set of web search results is an important kind of web resource that, like any other, should be URL-accessible. Unless I’m wrong, in which case someone please correct me, I can’t do the equivalent of `?q=notifications` in a Slack or Zendesk URL. Of course it’s possible to wrap their APIs in a way that enables URL query, but I shouldn’t have to. Every search deserves its own URL.

1 Thanks as always to Brewster Kahle for preserving what publishers did not.

Annotation-powered apps: A “Hello World” example

Workflows that can benefit from annotation exist in many domains of knowledge work. You might be a teacher who focuses a class discussion on a line in a poem. You might be a scientist who marks up research papers in order to reason about connections among them. You might be an investigative journalist fact-checking a claim. A general-purpose annotation tool like Hypothesis, already useful in these and other cases, can be made more useful when tailored to a specific workflow.

I’ve tried a few different approaches to that sort of customization. For the original incarnation of the Digital Polarization Project, I made a Chrome extension that streamlined the work of student fact-checkers. For example, it assigned tags to annotations in a controlled way, so that it was easy to gather all the annotations related to an investigation.

To achieve that effect the Digipo extension repurposed the same core machinery that the Hypothesis client uses to convert a selection in a web page into the text quote and text position selectors that annotations use to identify and anchor to selections in web pages. Because the Digipo extension and the Hypothesis extension shared this common machinery, annotations created by the former were fully interoperable with those created by the latter. The Hypothesis client didn’t see Digipo annotations as foreign in any way. You could create an annotation using the Digipo tool, with workflow-defined tags, and then open that annotation in Hypothesis and add a threaded discussion to it.

As mentioned in Annotations are an easy way to Show Your Work, I’ve been unbundling the ingredients of the Digipo tool to make them available for reuse. Today’s project aims to be the “Hello World” of custom annotation-powered apps. It’s a simple working example of a powerful three-step pattern:

  1. Given a selection in a page, form the selectors needed to post an annotation that targets the selection.
  2. Lead a user through an interaction that influences the content of that annotation.
  3. Post the annotation.

For the example scenario, the user is a researcher tasked with evaluating the emotional content of selected passages in texts. The researcher is required to label the selection as StrongPositive, WeakPositive, Neutral, WeakNegative, or StrongNegative, and to provide freeform comments on the tone of the selection and the rationale for the chosen label.

Here’s the demo in action:

The code behind the demo is here. It leverages two components. The first encapsulates the core machinery used to identify the selection. The second provides a set of functions that make it easy to create annotations and post them to Hypothesis. Some of these provide user interface elements, notably the dropdown list of Hypothesis groups that you can also see in other Hypothesis-oriented tools like facet and zotero. Others wrap the parts of the Hypothesis API used to search for, or create, annotations.

My goal is to show that you needn’t be a hotshot web developer to create a custom annotation-powered app. If the pattern embodied by this demo app is of interest to you, and if you’re at all familiar with basic web programming, you should be able to cook up your own implementations of the pattern pretty easily.

To work with selections in web pages, annotation-powered apps like these have to get themselves injected into web pages. There are three ways that can happen, and the Hypothesis client exploits all of them. Browser extensions are the most effective method. The Hypothesis extension is currently available only for Chrome, but there’s a Firefox extension waiting in the wings, made possible by an emerging consensus around the WebExtensions model.

A second approach entails sending a web page through a proxy that injects an annotation-powered app into the page. The Hypothesis proxy,, does that.

Finally there’s the venerable bookmarklet, a snippet of JavaScript that runs in the context of the page when the user clicks a button on the browser’s bookmarks bar. Recent evolution of the browser’s security model has complicated the use of bookmarklets. The simplest ones still work everywhere, but bookmarklets that load helper scripts — like the HelloWorldAnnotated demo — are thwarted by sites that enforce Content Security Policy.

It’s a complicated landscape full of vexing tradeoffs. Here’s how I recommend navigating it. The HelloWorldAnnotated demo uses the bookmarklet approach. If you’re an instructional designer targeting Project Gutenberg, or a scientific developer targeting the scientific literature, or a newsroom toolbuilder targeting most sources of public information, you can probably get away with deploying annotation-powered apps using bookmarklets. It’s the easiest way to build and deploy such apps. If the sites you need to target do enforce Content Security Policy, however, then you’ll need to deliver the same apps by way of a browser extension or a proxy. Of those two choices, I’d recommend a browser extension, and if there’s interest, I’ll do a follow-on post showing how to recreate the HelloWorldAnnotated demo in that form.

Annotating on, and with, media

At the 2018 I Annotate conference I gave a flash talk on the topic covered in more depth in Open web annotation of audio and video. These are my notes for the talk, along with the slides.

Here’s something we do easily and naturally on the web: Make a selection in a page.

The web annotation standard defines a few different ways to describe the selection. Here’s how Hypothesis does it:

We use a variety of selectors to help make the annotation target resilient to change. I’d like you to focus here on the TextPositionSelector, which defines the start and the end of the selection. Which is something we just take for granted in the textual domain. Of course a selection has a start and an end. How could it not?

An annotation like this gives you a kind of an URL that points to a selection in a document. There isn’t yet a standard way to write that URL along with the selectors, so Hypothesis points to that combination — that is, the URL plus the selectors — using an ID, like this:

The W3C have thought about a way to bundle selectors with the URL, so you’d have a standard way to cite a selection, like this:

In any case, the point is we’re moving into a world where selections in web resources have their own individual URLs.

Now let’s look again at this quote from Nancy Pelosi:

That’s not something she wrote. It’s something she said, at the Peter G Peterson Fiscal Summit, that was recorded on video.

Is there a transcript that could be annotated? I looked, and didn’t find anything better than this export of YouTube captions:

But of course we lack transcriptions for quite a lot of web audio and video. And lots of it can’t be transcribed, because it’s music, or silent moving pictures.

Once you’ve got a media selection there are some good ways to represent it with an URL. YouTube does it like this:

And with filetypes like mp3 and mp4, you can use media fragment syntax like this:

The harder problem in the media domain turns out to be just making the selection in the first place.

Here I am in that video, in the process of figuring out that the selection I’m looking for starts at 28:20 and ends at 28:36.

It’s not a fun or easy process. You can set the start, as I’ve done here, but then you have to scrub around on the timeline looking for the end, and then write that down, and then tack it onto the media fragment URL that you’ve captured.

It’s weird that something so fundamental should be missing, but there just isn’t an easy and natural way to make a selection in web media.

This is not a new problem. Fifteen years ago, these were the dominant media players.

We’ve made some progress since then. The crazy patchwork of plugins and apps has thankfully converged to a standard player that works the same way everywhere.

Which is great. But you still can’t make a selection!

So I wrote a tool that wraps a selection interface around the standard media player. It works with mp3s, mp4s, and YouTube videos. Unlike the standard player, which has a one-handled slider, this thing has a two-handled slider which is kind of obviously what you need to work with the start and end of a selection.

You can drag the handles to set the start and end, and you can nudge them forward and backward by minutes and seconds, and when you’re ready to review the intro and outro for your clip, you can play just a couple of seconds on both ends to check what you’re capturing.

When you’re done, you get a YouTube URL that will play the selection, start to stop, or an mp3 or mp4 media fragment URL that will do the same.

So how does this relate to annotation? In a couple of ways. You can annotate with media, or you can annotate on media.

Here’s what I mean by annotating with media.

I’ve selected some text that wants to be contextualized by the media quote, and I’ve annotated that text with a media fragment link. Hypothesis turns that link into an embedded player (thanks, by the way, to a code contribution from Steel Wagstaff, who’s here, I think). So the media quote will play, start to stop, in this annotation that’s anchored to a text selection at Politifact.

And here’s what I mean by annotating on media.

If I’m actually on a media URL, I just can annotate it. In this case there’s no selection to be made, the URL encapsulates the selection, so I can just annotate the complete URL.

This is a handy way to produce media fragment URLs that you can use in these ways. I hope someone will come up with a better one than I have. But the tool is begging to be made obsolete when the selection of media fragments becomes as easy and natural as the selection of text has always been.

Annotations are an easy way to Show Your Work

Journalists are increasingly being asked to show their work. Politifact does that with a sources box, like this:

This is great! The more citation of sources, the better. If I want to check those sources, though, I often wind up spending a lot of time searching within source articles to find passages cited implicitly but not explicitly. If those passages are marked using annotations, the method I’ll describe here makes that material available explicitly, in ways that streamline the reporter’s workflow and improve the reader’s experience.

In A Hypothesis-powered Toolkit for Fact Checkers I described a toolkit that supported the original incarnation of the Digital Polarization Project. More recently I’ve unbundled the key ingredients of that toolkit and made them separately available for reuse. The ingredient I’ll discuss here, HypothesisFootnotes, is illustrated in this short clip from a 10-minute screencast about the original toolkit. Here’s the upshot: Given a web page that contains Hypothesis direct links, you can include a script that pulls the cited material into the page, and connects direct links in the page to citations gathered elsewhere in the page.

Why would such links exist in the first place? Well, if you’re a reporter working on an investigation, you’re likely to be gathering evidence from a variety of sources on the web. And you’re probably copying and pasting bits and pieces of those sources into some kind of notebook. If you annotate passages within those sources, you’ve captured them in a way that streamlines your own personal (or team) management of that information. That’s the primary benefit. You’re creating a virtual notebook that makes that material available not only to you, but to your team. Secondarily, it’s available to readers of your work, and to anyone else who can make good use of the material you’ve gathered.

The example here brings an additional benefit, building on another demonstration in which I showed how to improve Wikipedia citations with direct links. There I showed that a Wikipedia citation need not only point to the top of a source web page. If the citation refers specifically to a passage within the source page, it can use a direct link to refer the reader directly to that passage.

Here’s an example that’s currently live in Wikipedia:

It looks like any other source link in Wikipedia. But here’s where it takes you in the cited Press Democrat story:

Not every source link warrants this treatment. When a citation refers to a specific context in a source, though, it’s really helpful to send the reader directly to that context. It can be time-consuming to follow a set of direct links to see cited passages in context. Why not collapse them into the article from which they are cited? That’s what HypothesisFootnotes does. The scattered contexts defined by a set of Hypothesis direct links are assembled into a package of footnotes within the article. Readers can still visit those contexts, of course, but since time is short and attention is scarce, it’s helpful to collapse them into an included summary.

This technique will work in any content management system. If you have a page that cites sources using direct links, you can use the HyothesisFootnotes script to bring the targets of those direct links into the page.

Investigating ClaimReview

In Annotating the Wild West of Information Flow I discussed a prototype of a ClaimReview-aware annotation client. ClaimReview is a standard way to format “a fact-checking review of claims made (or reported) in some creative work.” Now that both Google and Bing are ingesting ClaimReview markup, and fact-checking sites are feeding it to them, the workflow envisioned in that blog post has become more interesting. A fact checker ought to be able to highlight a reviewed claim in its original context, then capture that context and push it into the app that produces the ClaimReview markup embedded in fact-checking articles and slurped by search engines.

That workflow is one interesting use of annotation in the domain of fact-checking, it’s doable, and I plan to explore it. But here I’ll focus instead on using annotation to project claim reviews onto the statements they review, in original context. Why do that? Search engines may display fact-checking labels on search results, but nothing shows up when you land directly on an article that includes a reviewed claim. If the reviewed claim were annotated with the review, an annotation client could surface it.

To help make that idea concrete, I’ve been looking at ClaimReview markup in the wild. Not unexpectedly, it gets used in slightly different ways. I wrote a bookmarklet to help visualize the differences, and used it to find these five patterns:

  "urlOfFactCheck": "",
  "reviewRating": {
    "@type": "Rating",
    "ratingValue": "-1",
    "alternateName": "Fuzzy math",
    "worstRating": "-1",
    "bestRating": "-1",
    "image": ""
  "itemReviewed": {
    "@type": "CreativeWork",
    "author": {
      "@type": "Person",
      "name": "Donald Trump ",
      "jobTitle": "President",
      "image": "",
      "sameAs": [
    "datePublished": "2018-03-13",
    "name": "in a tweet"
  "claimReviewed": "\"The $18 billion wall will pay for itself by curbing the importation of crime, drugs and illegal immigrants who tend to go on the federal dole.”"

Did a Captured Islamic State Leader Have Barack Obama On Speed Dial?

  "urlOfFactCheck": "",
  "reviewRating": {
    "alternateName": "False",
    "ratingValue": "",
    "bestRating": ""
  "itemReviewed": {
    "datePublished": "2018-03-13T08:25:46+00:00",
    "name": "Snopes",
    "sameAs": ""
  "claimReviewed": "A captured Islamic State leader's cell phone contained the phone numbers of world leaders, including former President Barack Obama."

Trump Misses with Mars Attack

  "urlOfFactCheck": "",
  "reviewRating": {
    "@type": "Rating",
    "ratingValue": "-1",
    "alternateName": "Clinton Endorsed Mars Flight",
    "worstRating": "-1",
    "bestRating": "-1",
    "image": ""
  "itemReviewed": {
    "@type": "CreativeWork",
    "author": {
      "@type": "Person",
      "name": "Donald Trump",
      "jobTitle": "President of the United States",
      "image": "",
      "sameAs": [
    "datePublished": "2018-03-13",
    "name": "Speech at military base"
  "claimReviewed": "NASA “wouldn’t have been going to Mars if my opponent won.\""

  "urlOfFactCheck": "",
  "reviewRating": {
    "@type": "Rating",
    "ratingValue": "2",
    "alternateName": "False",
    "worstRating": "1",
    "bestRating": "7",
    "image": ""
  "itemReviewed": {
    "@type": "CreativeWork",
    "author": {
      "@type": "Person",
      "name": "Ted Cruz",
      "jobTitle": "U.S. senator for Texas",
      "image": "",
      "sameAs": [
    "datePublished": "2018-03-06",
    "name": "Houston, Texas"
  "claimReviewed": "Says Beto O’Rourke wants \"open borders and wants to take our guns.\""

Breitbart repeats blogger’s unsupported claim that NOAA manipulates data to exaggerate warming

  "urlOfFactCheck": "",
  "reviewRating": {
    "@type": "Rating",
    "alternateName": "Unsupported",
    "bestRating": 5,
    "ratingValue": 2,
    "worstRating": 1
  "itemReviewed": {
    "@type": "CreativeWork",
    "author": {
      "@type": "Person",
      "name": "James Delingpole"
    "headline": "TBD",
    "image": {
      "@type": "ImageObject",
      "height": "200px",
      "url": "",
      "width": "200px"
    "publisher": {
      "@type": "Organization",
      "logo": "Breitbart",
      "name": "Breitbart"
    "datePublished": "20 February 2018",
    "url": "TBD"
  "claimReviewed": "NOAA has adjusted past temperatures to look colder than they were and recent temperatures to look warmer than they were."

To normalize the difference I’ll need to look at more examples from these sites. But I’m also looking for more patterns, so if you know of other sites that routinely embed ClaimReview markup, please drop me links!