Jon Udell

The fulcrum of my talk last week at the Open Education Conference was observable work. I first started thinking about this back in 2002, when I included this Dave Winer excerpt in my review of Radio UserLand:

We’ve been using this tool since November, internally at UserLand. We shipped Radio 8 with it. When we switched over our workgroup productivity soared. All of a sudden people could narrate their work. Watch Jake as he reports his progress on the next project he does. We’ve gotten very formal about how we use it. I can’t imagine an engineering project without this tool.

Since then I’ve spoken a few times about the idea that by narrating our work, we can perhaps restore some of what was lost when factories and then offices made work opaque and not easily observable. Software developers are in the vanguard of this reintegration, because our work processes as well as our work processes are fully mediated by digital networks. But it can happen in other lines of work too, and I’m sure it will.

My favorite example, from a very different domain, is the historic home preservationist John Leeke. In our interview he eloquently explains how and why he works observably.

This week’s Innovators show, with Charlie O’Donnell and Hilary Mason of Path101, expands on the same theme from a different perspective. Path101′s tagline is community-powered career discovery, and the approach is more data-driven than narrative.

When we narrate our work, we enable others to ask and answer the critical question:

What is it like to be a __________?

Path101′s aggregation of resumes and personality tests aims for different kinds of questions:

What personality traits do other _______s like me tend to have?

What careers do other _______s like me transition into?

Path101 is still a very young service, but I love the concept and will be interested to see how it evolves.

I spent last weekend in DC at Transparency Camp, which turned out to be one of the best cultural mashups I’ve attended in a long time. If we can get federal policy wonks and Silicon Valley tech geeks working together in the right ways, there’s good reason to hope that our government can become not just more transparent, but also more effective, more collaborative, more democratic.

A central theme was access to the operational data of government. What kinds of structured or narrative data exist, or could exist? When government doesn’t publish the stuff, how can activists extract it? When government does publish it, how can that be done most usefully? When the information is made available, one way or another, how can citizens, journalists, and government itself make use of it?

In my own work, I’ve been asking and trying to answer these questions. The event validated my efforts, and connected me to a flood of relevant people, ideas, tools, and techniques. That’s what you hope to get out of a conference, and it’s what this one delivered in spades.

But it also brought something else into sharp focus. To explain, I have to revisit 1994. In that seminal year, Microsoft famously “got” the Web. As BusinessWeek reported two years later:

The Web-izing of Microsoft begins in February, 1994, when Steven Sinofsky, Gates’s technical assistant, returned to his alma mater, Cornell University, on a recruiting trip. Snowed in at the Ithaca (N.Y.) airport, he headed back to the Cornell campus. That’s when he saw it: students dashing between classes, tapping into terminals, and getting their E-mail and course lists off the Net.

The Internet had spread like wildfire. It was no longer the network for the technically savvy — as it had been seven years earlier when Sinofsky was studying there — but a tool used by students and faculty to communicate with colleagues on campus and around the world. He dashed off a breathless E-mail message called “Cornell is WIRED!” to Gates and his technical staff.

Fifteen years on, the Net is as pervasive as air, as fundamental as gravity, as nourishing as sunlight — at least for the billion of us lucky enough to be online.

But while the architecture of the Net is firmly established, the architecture of communication and collaboration enabled by the Net is still very much up for grabs. Key principles, best practices, and effective patterns are still emerging.

For many years I have been a discoverer, early adopter, and explainer of those principles, practices, and patterns. And I’ve wondered: What would it would be like if you didn’t have to discover, adopt, and explain this stuff? What would it be like if you could just take it for granted, and just use it, in an environment where everybody else was using it too?

It would be like Transparency Camp 09.

This wasn’t the first event I’ve been to where Twitter was pervasive. But it was the first I’ve been to where tech geeks weren’t the only ones Twittering. The policy wonks were too. Everyone was tuned into the #tcamp09 channel. And, in fact, everyone still is. The conference “ended” on Sunday, it’s Thursday, but a half-dozen new items appeared on that channel since I started writing this essay. I particularly like this one:

Funny. Someone from #tcamp09 lives in my building. She says, “Didn’t we meet this weekend?” “No.” “You’re…cheeky something?” “OH…yes”

That’s a nice example of manufactured serendipity. I coined the phrase in another era. Back then, the new phenomenon called blogging was the realm in which we were discovering, adopting, and explaining the crucial principles, patterns, and practices. Now the action has moved to Twitter. But they’re the same principles, patterns, and practices:

1. The principle of conserving keystrokes
2. The pattern of publishing and subscribing
3. The practice of narrating your work

In 1994 Steve Sinofsky saw the arrival of the Net, and sent email to tell Microsoft about it. In 2009 I see the emergence of a transformative way of using the Net. I could try sending email to tell Microsoft about it, and that would still be the preferred method. But email is no longer the engine that will drive radical improvement. What’s more, it often subverts the right principles, patterns, and practices.

So how does Microsoft, or any large enterprise — e.g., the government — embrace a new architecture of communication and collaboration? Slowly at first, but inexorably, and with profound effects in the long run. I can’t alter the timetable. But this is an interesting moment, and I simply want to observe, mark, and note it.

I was in Montreal on Saturday to give a talk at CUSEC 2008, a great conference that’s organized by Canadian software engineering students (and recent grads) who want to congregate, exchange views, and hear from speakers they think will provide useful insight.

I gave the morning keynote, and Jeff Atwood spoke in the afternoon. Our messages dovetailed in an interesting way. Jeff’s title was: “Is writing more important than programming?” As a wildly successful blogger, the influence of his own writing has eclipsed the influence of his programming. He admits that this result is “not typical,” but argues that every programmer will reap benefits from narrating the work: influence, collaboration, clarity of thought.

My title was “Hacking the Noosphere”. The themes are shared tools and data, social engineering, language, the semantic web, human augmentation, and Doug Engelbart’s vision of the true purpose of information technology.

Although I’m trying to be more extemporaneous these days, I had a lot I wanted to say, and I wanted to say it carefully, so this turned into one of those talks that I wrote out completely. The upside is that I can make it available to read.

I am an immediate fan of Common Craft’s style of concept videos. Their explanations of how and why to use del.icio.us and Google Docs are crisp and entertaining. They convey the essence of these activities more clearly than any other visual explanations I’ve seen, including many of the screencasts I’ve made.

The style is called paperworks because these sketchcasts are made by capturing screenshots, printing out key elements, and then filming, animating, annotating, and narrating arrangements and rearrangements of these scraps of paper. The first time you watch one, you’ll be captivated: it’s cute, it’s fresh. But is this just a gimmick? After you watch a few more, and you begin to acclimate to the style, does its effectiveness wane? Not yet, for me, because these productions have more going for them than cuteness and freshness.

One of the principles at work here is the moral equivalent of cropping and zooming in the screencast medium. When you’re trying to explain software on a conceptual level, images captured from screens can be a mixed blessing. It’s valuable to show exactly what screens look like, and exactly how actions flow within and across them. But the amount of detail that’s visible in a typical screen can often distract from the story you’re trying to tell. By cropping the screen, and/or by zooming in on the active region, you can prune away a lot of visual clutter and focus on key interactions. The paperworks style is an extreme form of cropping and zooming; it prunes and focuses very aggressively.

Another principle is sketching. According to Bill Buxton, sketching goes hand in hand with what he calls design thinking. When I asked Bill how he would have used sketching in the design of a feature like the Office ribbon, he said:

You’d start with paper prototyping — quickly hand-rendered versions, and for the pulldown menus and other objects you’d have Post-It notes. So when somebody comes with a pencil and pretends it’s their stylus and they click on something, you’ve anticipated the things they’ll do, and you stick down a Post-It note.

If that’s a helpful way to imagine software interaction in the design phase, why wouldn’t it also be a helpful way to conceptualize the software in use? The paperworks style strongly suggests that it is. These sketchcasts are great visual explanations of working software. I suspect they’d be equally useful during the design of that software.

When I read this story about cancer care in the Sunday New York Times yesterday, I was struck by one particular information graphic which I thought was very nicely done:

It turns out that Chris Gemignani was impressed too, and he decided to recreate the image using Excel. Here’s what he came up with:

Going one huge step further, and in the spirit of today’s theme of narrating the work, he created a screencast in which he demonstrates the process of making that graphic. It’s a wonderful example of the dynamic I’ve been describing. One of the commenters on Chris’ blog thanks him for teaching him some helpful techniques. Another suggests a technique that Chris hadn’t used but thinks is interesting. Very cool!

With Excel, as with all software — on the desktop and on the web — there’s so much untapped potential. The obstacles are knowing what’s even possible, and then knowing how to achieve it. Screencasts like this one leap over both obstacles in a single bound.

Last year Greg Wilson wrote to tell me about the collection of essays that he and Andy Oram were compiling into what has now become the book Beautiful Code: Leading Programmers Explain How They Think:

The idea is to get a bunch of well-known and not-yet-well-known programmers to select medium-sized pieces of code (100-200 lines) that they think are particularly elegant, and spend 2500 words or so explaining why.

The 600-page tome arrived recently, and as I’ve been reading it I’m struck once again by the theme of narrating the work. Of the chapters I’ve read so far, three are especially vivid examples of that: Karl Fogel’s exegesis of the stream-oriented interface used in Subversion to convey changes across the network, Alberto Savoia’s meditation on the process of software testing, and Lincoln Stein’s sketches (“code stories”) that he writes for himself as he develops a new bioinformatics module.

Although this is a book by programmers and for programmers, the method of narrating the work process is, in principle, much more widely applicable. In practice, it’s something that’s especially easy and natural for programmers to do.

It’s easy because a programmer’s work product — in intermediate and final form — happens to be lines of text that can be printed in a book or published online.

It’s natural because programmers have been embedded for longer than most other professionals in a work process that’s fundamentally enabled by electronic publishing. We’ve been sharing code, and conversations about code, online for decades.

Most work processes don’t lend themselves to the sort of direct capture and literal representation that you see in Beautiful Code. Not yet, anyway. I think that can and will change, though, and I think two emerging forms of media will be powerful agents of change.

One of those forms is Internet video, which enables the capture and sharing of many kinds of physical-world expertise. The other is screencasting, which does the same for virtual-world expertise. Narration of work in these forms won’t be able to be printed in a book. But it will be just as valuable as the narration in Beautiful Code, and for the same reasons. Access to expert minds is just inherently valuable. We’re entering an era in which we’ll be able to access many more — and many different kinds of — expert minds. I’m looking forward to it. Meanwhile, I’m enjoying the access I have now to the 38 minds that Greg and Andy have collected for this book.

As director of web publishing for Nature Publishing Group, Timo Hannay’s projects include: Connotea, a social bookmarking service for scientists; Nature Network, a social network for scientists; and Nature Precedings, a site where researchers can share and discuss work prior to publication.

The social and collaborative aspects of these systems are, of course, inspired by their more general counterparts on the web: del.icio.us, Facebook and LinkedIn, the blogosophere. That’s part of what we discussed in this week’s ITConversations podcast. We also talked about my longstanding concern that scientists, like other academics and indeed most professional people, aren’t directly rewarded for being wired into the web. Timo has some great ideas about how to change that. He notes:

This will sound a bit strange coming from someone who works for a journal publisher, but to date, the way that scientists’ output has been measured has been unduly focused on publications in peer-reviewed journals. That is, and will continue to be, a really important part of it, but it’s not the only thing they do.

Here’s one specific proposal for change — measure, and reward, contributions of data:

Biology in recent years has seen a move from what I would characterize as cottage industry science, where everything from data capture through to analysis to writing the paper happens within one lab among a small group of people, to a much more industrial scale where you have different groups, widely dispersed, perhaps who don’t even know each other, doing the data capture versus the analysis versus writing the paper.

But you can’t just publish a data set. So what tends to happen is that, for a really big important data set — like a new major genome — they’ll publish a paper off the back of it, and do a very quick preliminary analysis. But the real news is not the analysis, it’s the data set. They have to make this fig leaf of analysis in order to justify publishing the paper.

We need to make it possible for people to publish data sets — to put them out there, track what use is made of them by other people, and then eventually gain credit for that.

Excellent suggestion!

More broadly, Timo wants to measure activity in the specialized versions of the blogging, bookmarking, and social networking services that Nature Publishing Grouop is creating for scientists. He says NPG is working with funding organizations to figure out what kinds of measurement can support a broader system of credit and recognition.I know it’s hard to nail down this touchy-feely stuff, but it really does matter. Yesterday I found a great quote from E.O. Wilson — in Consilience, which I’ve finally gotten around to reading — that helps explain why:

The creative process is an opaque mix. Perhaps only openly confessional memoirs, still rare to nonexistent, might disclose how scientists actually find their way to a publishable conclusion. In one sense scientific articles are deliberately misleading. Just as a novel is better than the novelist, a scientific report is better than the scientist, having been stripped of all the confusions and ignoble thought that led to its composition. Yet such voluminous and incomprehensible chaff, soon to be forgotten, contains most of the secrets of scientific success.

Narrating the work in openly confessional memoirs can and should be measurable, valuable, credit-worthy.

For this week’s ITConversations show I talked with Tessa Lau about Project Koala, a “a system for recording, automating, and sharing business processes performed in a web browser.” I’ve been interested in that idea for a long time, and mentioned it most recently in this item on pooling citizens’ collective knowledge about the services of government websites, and about how to make effective use of those services. In a comment on that item, Koranteng Ofosu-Amaah mentioned Project Koala and suggested that I speak with Tessa about it, so I did.

Of course we’ve had macro recorders since the dawn of computing, and Koala is yet another of those. What’s different? Crucially, the ability not only to capture and replay, but also to share, performances of tasks. The descriptions of those tasks are shared on a Wiki, and they’re written in an English-like syntax that’s very close to what you’d write if you were narrating instructions, e.g.: “Enter 94301 into the Search By Zipcode textbox, then click the Continue button.” These instructions can be edited, tagged, searched, and indexed by the URLs embedded in them. In theory that will enable us to pool our experiential knowledge of web applications. In practice, we’ll see. Koala has yet to emerge from IBM’s research lab. But you’ve got to love the idea.