Jon Udell

The WSJ reported recently that the FBI, looking for fresh leads in the 1982 case of Tylenol poisonings, suspects Ted “Unabomber” Kaczynski and is trying to get hold of a sample of his DNA. Coincidentally I was just thinking about that case thanks to Bruce Schneier. In his recent TED talk he mentions that the Tylenol incident led to tamper-proof caps — a perfect example of what Schneier likes to call “security theater”:

As a homework assignment, think of 10 ways to get around it. I’ll give you one, a syringe.

So far this is typical Schneier. It’s a great point, but one I’ve heard him make many times before. In the next sentence, though, he breaks new ground:

But it made people feel better. It made their feeling of security more match the reality.

Bruce Schneier used to mock the theatrical dimension of security. Now it seems his thinking has evolved — and in a really interesting way. He’s alway viewed security in a relativistic way, and as a game of economic tradeoffs. Here he twists the lens to bring something else into focus: the relationship between how secure we feel and how secure we are.

We know that modern hominids evaluate risk poorly:

One way to think of it is that we’re highly optimized for risk decisions that are endemic to living in small family groups in the East African highlands in 100,000 B.C. — 2010 New York, not so much.

There are two ways our feelings about our security can disconnect from reality. We can have a false sense of security, as when we underestimate the risk of automobile travel. Or we can have a false sense of insecurity, as when we overestimate the risk of air travel.

What matters most, Schneier suggests, is that we align feelings and reality. And if security theater helps us do that, then it’s a perfectly good thing. The Tylenol episode represents the kind of risk that the media thrive on and magnify. People felt unsafe even though they were astronomically more likely to die in other ways. Tamper-proof caps may be a purely theatrical response, but that may still be useful.

Technologists too rarely display this kind of nuanced thinking; it’s refreshing to see it.

The elmcity project thinks like the web, and invites users to do the same. Direct users of the elmcity service (hereafter just elmcity) are curators who run calendar syndication hubs. Indirect users are people who contribute calendar feeds that flow through the hubs. Everyone involved in this dance practices key principles of web thinking. Here are some ways:

In these scenarios the elmcity service partners with a number of other services, most notably Delicious. During last December’s Delicious-is-going-away scare I compiled a list of essays I’ve written since 2004 about how and why Delicious is far more useful than the tagline “social bookmarking” might suggest. Here’s the gist: Delicious, more than any other online service I know, encourages and rewards the practice of web thinking. That makes it a natural partner of elmcity, a service that invites communities to think like the web.

There are, of course, other ways to manage hub settings and feed registries. Various social bookmarking services can do what Delicious does. More conventionally, the elmcity service could manage this data directly. I’ve always been prepared to swap out Delicious for something else, and when it looked like Delicious might go away I reviewed my options.

Then came news that Delicious had been acquired by YouTube’s founders and would continue as part of a new company called AVOS. The transition promises to be seamless. In order to continue using their existing accounts and data after July 2011, Delicious users need only opt in to the AVOS terms of service.

For typical users it’s all straightforward, though you’ll want to note that AVOS declines responsibility for backing up your contributed data. I’m glad to see that in writing. Too many people assume, wrongly, that the data they send to free cloud services, in order to socialize with other people and other people’s data, will be backed up. It’s the cloud, right? The cloud can back up your stuff, right? Yes it can, but don’t expect that to happen unless you’re paying for it.

The elmcity service isn’t a typical user of Delicious, though. It accesses Delicious programmatically, scanning curators’bookmarks several times a day in order to check for updates to their hub settings and feed registries. That may not square with this item from the list of Things Not To Do in the AVOS terms of service:

Attempt to access or search the Service or Service Content or download Service Content from the Service through the use of any engine, software, tool, agent, device or mechanism (including spiders, robots, crawlers, data mining tools or the like) other than the software and/or search agents provided by AVOS or other generally available third party web browsers.

That’s quite a change from the comparable paragraph in the existing terms of service:

Delicious provides access to portions of Delicious via RSS feeds and an API; for the purposes of these Delicious Terms, such access constitutes use of Delicious. Delicious asks that you use these features respectfully, as outlined in the documentation.

Elmcity actually makes very little use of the Delicious API. But it relies heavily on the fact that for every HTML page that Delicious presents, there is a corresponding RSS feed. For example, curators add iCalendar feeds to their hub registries by bookmarking the URLs of those feeds, and tagging them in a certain way. As curator of the hub for Keene, NH, I review my hub’s registry using this Delicious query:

http://delicious.com/elmcity/trusted+ics+feed

When the elmcity service checks for updates to my registry, it uses the corresponding RSS feed:

http://feeds.delicious.com/v2/elmcity/trusted+ics+feed

That same feed serves another purpose. The elmcity support forum and project collaboration space is on FriendFeed. One of FriendFeed’s endearing features is ad-hoc RSS aggregation for groups. The elmcity group on FriendFeed compiles project status from a variety of RSS feeds, including Delicious feeds of the form shown above. What that means is that when any curator discovers a new iCalendar feed and adds it to his or her hub, everyone in the FriendFeed space is notified. It’s a wonderful example of how pub/sub networks can enable ambient awareness.

I hope that AVOS wants to celebrate, not forbid, this kind of emergent capability. But it does require accessing the service with tools and services other than “generally available third party web browsers.” I have always done this kind of access respectfully, and would like to continue on that basis. But if that won’t be acceptable I’ll make other arrangements. Either way I want to thank Delicious for many years of stalwart service and creative stimulation, and to thank AVOS for carrying on. I’m sending the Delicious/AVOS support team a link to this post, and I’ll use comments here to report what I find out.

I love small airports. When we moved to Keene, NH, over 20 years ago, I could roll out of bed, drive three minutes to our town’s airfield, hop onto a flight to LaGuardia, arrive in midtown Manhattan for a day of meetings, and be home for a late dinner. It was a good thing that didn’t last. The federal subsidy that kept those flights going ran dry, and Keene’s commercial air service ended long ago.

Years later I read James Fallows’Free Flight and fell in love with its vision of an alternative air travel system that thought like the web: a decentralized and fluid network of interoperable resources.

Still later an old acquaintance, Ed Iacobucci, made a valiant effort to bring that vision to life. Our 2007 conversation about DayJet, the on-demand regional jet travel service Ed had then just launched, inspired me to hope that I might again fly to and from the Keene airport. Our hopes were dashed by the 2008 economic collapse, and now that vision is on hold. But last night I had a taste of what air travel once was and might someday be again.

I’m in Toronto for a couple of days. On my last two visits, not knowing another way, I used Toronto’s international airport, Pearson. It’s a typically unfriendly major hub, made even less friendly by its atypical lack of rail service into the city center. But on my last trip, during a morning lakeside run, I saw small planes landing at the city airport on Toronto Island within a stone’s throw of downtown. I wondered what it would be like to arrive on one of those flights.

Well, now I know. It’s delightful! I flew Porter Airlines from Boston Logan direct to Toronto Island Airport. It was a trip from a bygone era. The plane was a turboprop. The complimentary wine was served in a real glass. The customs queue was quick. The ferry crossing to downtown, over a hundred yards of water, was seamless. Then I simply walked, less than a mile, to my downtown hotel. True, it cost a hundred bucks more than flying into Pearson. But I saved almost that much money, plus a chunk of time, by not cabbing from Pearson. And I arrived refreshed and happy. When does that ever happen?

Porter Airlines isn’t the kind of service that James Fallows’book imagined and Ed Iacobucci’s company created. But the retro 1950s-era experience that Porter offers may help to remind us that small airports are everywhere, waiting to offer pleasure and convenenience once again — if we can find ways to reactivate them.

I recently asked a web server for something and it responded like a teenager:

403 Forbidden: The server understood the request, but is refusing to fulfill it.

I thought the description of this status code was a joke, but no, it’s right there in the HTTP specification. Here are some others that you won’t find there:

200 OK: Are you happy now?

202 Accepted: I’ll do it tomorrow.

206 Partial Content: I’ll do the rest of it tomorrow.

301 Moved Permanently: It’s not my job.

305 Use Proxy: I can’t remember, ask mom.

401 Unauthorized: You’re not the boss of me.

404 Not Found: Why does everybody always ask me for that?

410 Gone: Zzzzzzzzzzzzzz.

417 Expectation Failed: I thought you were going to put gas in the car.

500 Internal Server Error: I’m not allowed to make a mistake?

The most interesting app on my phone at the moment is called PowerNow. Here’s the display:

At 12:14AM today, when I took this snapshot, my house was using 1090 watts. The data comes from the TED5000 smartmeter I recently installed. It reports all kinds of data but that single number matters most. When you can monitor your power use in realtime, you can regulate it far more intelligently than you can if your only instrument is a monthly bill.

The other day, for example, I finally got around to swapping out a bunch of incandescent bulbs for compact fluorescents. I took a baseline measurement with PowerNow, and then swapped out a total of 17 bulbs in six batches. After each batch I took another measurement. You can see all the data here. Step by step the wattage drops, by a total of 900. And step by step the hourly cost drops, by a total of $0.15/hr. Of course I won’t really save $0.15/hr because I took the baseline with everything turned on, which isn’t normally the case. But it’s wonderfully motivating to be able to measure the effect directly.

The TED5000 comes with a wireless display that I could have used to take those snapshots, but its ZigBee controller isn’t working well and I’m awaiting a replacement. So I turned to Google PowerMeter, which can receive data posted from the TED. But PowerMeter only reports cumulative use, not realtime use. When I put up my own experimental service to receive TED data, I found out why: cumulative data is all that the TED sends.

There are actually two TED APIs, though. The one used by PowerMeter, and by other services like the one I wrote, requires an activation protocol and then accepts data packets sent from the TED gateway. In other words: TED pushes to the service.

There’s also another API that enables a service to pull data from TED. It’s just this:

http://ted5000/api/LiveData.xml

This URL refers to a TED gateway that’s hooked into a home network which resolves the name ted5000 to some private address like 192.168.2.100. There are variant URLS for fetching historical data but this URL just brings back the realtime data which is all I want for now.

One way to view that data from anywhere would be to open up an inbound port on the router. But I’d rather not, so instead I’m running a scheduled process on one of my networked PCs. It polls the TED, extracts the one number I want, and posts it to a web page along with the date and time.

If I were in marketing, here’s how I’d describe this “cool app”:

Monitors realtime power use in your home!

Works anywhere, anytime!

Compatible with all smartphones, including iPhone, Android, BlackBerry, and Windows Phone!

And that’s all true. But of course it isn’t really an app, it’s just a web page backed by a suite of cooperating services: the TED’s software, my data extractor, and the website that hosts the page.

Or is it, in fact, an app? For me the word connotes not just software that runs on your phone, but rather software that runs in various places including your phone, and ties together various services.

Apps defined that way can be ridiculously easy to write. I tweeted that PowerNow is just 6 lines of Python. I went back and checked and, well, it’s a bit more, but not much more:

import urllib2, re, ftplib, datetime
from ftplib import FTP
from datetime import datetime

url = 'http://ted5000/api/LiveData.xml'
s = urllib2.urlopen(url).read()
power = re.findall('<PowerNow>(\d+)</PowerNow>', s)[0]

html = """<html>
<head><title>PowerNow</title></head>
<body>
<p style="font-size:x-large;text-align:center">%s</p>
<p style="text-align:center">%s</p>
</body>
</html>""" % (power, datetime.now().strftime('%A %B %d %H:%M'))

fname = '/users/jon/dev/PowerNow.html'

f = open(fname, 'w')
f.write(html)
f.close()

ftp = FTP('_______.net')
ftp.login('_______','________')
f = open(fname)
ftp.storbinary('STOR PowerNow.html', f)
f.close()
ftp.quit()

That’s the script that polls the TED, extracts the current power use in watts, packages it in HTML, and posts it to a webserver. To run it, I scheduled a once-per-minute process using the Windows 7 Task Scheduler. I hadn’t used Task Scheduler in a while, and I noticed something non-obvious about it. When you specify repetition, it looks like you only have these choices:

What’s non-obvious is that you can edit the values in the list, like so:

I also chose the One time setting:

That’s how you set up a task to repeat with arbitrary frequency. Then you tell it what to do. In my case:

I avoid home automation projects that require soldering, it just isn’t my thing. But stitching web componentry together? I can do that in my sleep. I’m really looking forward to the coming wave of appliances that will enable me to do that.

When I wrote about my experience with the Kill-A-Watt, commenters alerted me to the next level in the real-time power monitoring game: gadgets that watch the whole house instead of one appliance at a time. Back then the only solutions I could find were for UK power systems, not US power systems. But recently I found The Energy Detective and ordered a TED 5000 straightaway.

The bottom line is that it works. I’ve got a real-time display showing how many watts my house is burning at any given time, along with the equivalent hourly cost. When I turn off lights or appliances that don’t need to be on, I can see the difference. There’s much more data available, but continuous instant feedback is the essential thing. It’s a powerful behavior changer.

Unfortunately my TED system isn’t yet working as well as it should. One of the components is ailing. It’s the gateway that receives data from the transmitter I put in my distribution panel, and sends it along to the wireless display and to my LAN. The gateway’s local webserver won’t respond to requests, and the gateway’s connection to the wireless display is flaky. So I’ve ordered a replacement. I hope it’ll work properly, and if it does I’ll have more to say about the software aspects of the TED. But meanwhile, this is a good time to talk about the basic hardware setup. It’s pretty straightforward, but I think the vendor, Energy Inc., explains it misleadingly. That’s a shame. I hope lots of people will be able to use this remarkable tool, and I want them to have good experiences with it. So here’s what I think Energy Inc. does wrong with its installation guide, and how I think it could be improved.

The guide says:

A technically-savvy homeowner, neighbor, friend or electrician can install TED in 10-15 minutes.

And that’s true. But there are different ways to be technically savvy. The TED is a mashup of two different technologies: electrical distribution and data networking. Each has its own set of concepts and associated terminology. An electrician may know little about data networking. A homeowner like me may know a lot about data networking but very little about electrical distribution. A map of concepts and a glossary of terms would be really helpful.

On the electrical side, the terminology includes 120/240V single-phase, A phase, B phase, two-pole breaker, and handle tie. If you’re an electrician it won’t occur to you that the terms single-phase, A phase, and B phase will confuse somebody who isn’t an electrician. Like me. If it’s a single-phase system, I wondered, how is there an A phase and a B phase? I had to consult Wikipedia’s article on split-phase electric power:

A transformer supplying a 3-wire distribution system has a single-phase input (primary) winding. The output (secondary) winding is center-tapped and the center tap connected to a grounded neutral. This 3-wire system is common in countries with a standard phase-neutral voltage of 120 V. In this case, the transformer voltage is 120 V on either side of the center tap, giving 240 V between the two live conductors, shown as V1 and V2 in Fig. 1. The two outputs are properly called “legs”, not “phases”.

Ah. Now I could see why the TED’s MTU (measuring transmitting unit) needs (I wrongly thought) to be connected to the A “phase” and the B “phase” — all the wiring in my house is divided into two “legs” and the MTU (I wrongly thought) needs to reach both of them. Here’s the recommended procedure:

Step 3.

A.) Connect the black and red wires from the MTU power cord to a spare 15, 20, or 30 amp two-pole circuit breaker.

B.) If there is no spare circuit breaker in the panel, it can be attached to any 15, 20, or 30 amp two-pole breaker in the panel. If a two-pole breaker is not available, then use an approved handle-tie to create one.

At this point, as a data-network-savvy but electrical-distribution-naive homeowner, I’m ready to call in an electrician. It’s lucky I didn’t, though, because a data-network-naive electrician would likely have missed the significance of this cryptic remark stuck in between steps A and B:

(Option: For increased signal strength, connect only the black wire.)

Huh? Oh. Now my data networking knowledge kicks in. The MTU uses power line networking. That means it sends data over electrical wiring. Where does it send data? To the gateway, a little box that plugs into an outlet somewhere and talks to the wireless display and also to a computer or, if you run a local area network, to your Ethernet router. The recommended place to plug in the gateway is an outlet near your router, so you can control it and read data from it using any computer on your LAN.

The instructions tell you to connect the MTU to both legs of your electrical distribution. In theory that enables you to plug the gateway into any outlet in the house. In practice, as you’ll find if you try, fail, and then consult the troubleshooting guide, probably not.

But now I began to see that you don’t need to do that. You could connect just one of its wires to one breaker that controls an outlet near your router. And in fact that’d be better, because power line communication can be flaky and you want to use the most direct signal path. (The outlet should preferably be near your router but not actually on a circuit that powers noisy devices like the router, or computers, or the TV. And I found one that met that requirement.)

I found confirmation for this direct approach in the troubleshooting guide:

3.7. Considering the factors affecting PLC communication, if the Gateway is still not receiving a signal, do the following:

3.7.1. Connect the black wire of the MTU to the circuit on which the Gateway is connected.

3.7.2. Remove the red wire from the circuit in which it is connected, and put a wire nut onto the red wire.

Further confirmation came from the tech support guy I spoke with when I reported the sick gateway. “Yeah, I don’t know why the manual says to use a spare two-pole breaker,” he told me. “A lot of times that doesn’t work, then people call me up and I tell them to do it the way you did.”

Documentation that spelled out the concepts and terminology in each domain, and the relationships between the domains, would help people grounded in one or the other of the domains see the whole picture.

On the data networking side, the terminology includes PLC (power line communication), Ethernet, and ZigBee. There are actually three kinds of data networking happening. The MTU talks PLC to the gateway. The gateway talks Ethernet to your computer or, if you’re running a local area network as many now do, to any computer on the LAN. And the gateway talks ZigBee to the TED handheld wireless display. All this was obvious to me, but might not be to an electrician or electrically-savvy homeowner. Spelling out concepts and terminology would help that person sort out what gets distributed where:

- Electricity to the whole house, via the two legs of the power system

- Data to the gateway, via the electrical wiring

- Data to/from a computer or router via Ethernet cable

- In a LAN situation, data to/from one or more computers via Ethernet cable and/or WiFi

- Data to the wireless display via ZigBee

Maybe a lot of this context was provided by other TED users in the support forum. Except, oops:

Several weeks ago our Forum Server crashed, and we were unable to recover the valuable library of comments that our users had posted over the prior year.

Those forum members have now learned, the hard way, to appreciate the first of the seven ways to think like the web:

1. Be the authoritative source for your own data

The Energy Inc. site goes on to say:

Our new Forum Server has built-in safeguards to prevent the loss of data should there be a hardware malfunction. We do encourage those of you who have been avid online supporters to please continue to assist those new TED users in their quest to become experts!

Sure, happy to do it, but not by posting this article to the forum. Instead I’ll invoke the second Thinking Like The Web principle:

2. Pass by reference not by value

I want this article to help new TED users. Maybe somebody in the new forum will link here. Maybe I’ll do it myself if I can be bothered to create Yet Another Account. But even if that doesn’t happen, there’s an implicit connection via search. Anybody searching for key terms in this article, along with TED 5000, will land here.

I’m glad the forum server has backup now. But I still don’t want to commit a lot of my own keystrokes to somebody else’s cloud database. I want to keep those keystrokes in a cloud database that’s accountable to me, and then link them to wherever else they need to show up.

One of the first posts I wrote with Radio UserLand, my original blogging tool, was this one from early 2002:

Messages addressed to spaces

I just realized why the idea of messages addressed to spaces hit me so hard. Email is a message addressed to a person or group, where as blogging (or posting to a newsgroup or web forum) is a message addressed to a space. A group may or may not form at the coordinates of that space. If a group does form, people may be there at roughly the same time, or may visit serially, separated by hours or days or even years. Why does this seem so special and important to me? I’m still not completely sure, I only know that it does.

Remarkably that blog post still survives at radio-weblogs.com. The New Scientist story it cited, though, was a casualty of a content management regime change. But I remember the story. It described a scenario that was futuristic then. You’re walking around in New York City with a handheld device that knows your location; you can send messages tagged with your present coordinates; you can also receive messages tagged with your present coordinates. The physical world becomes a bulletin board carved up into an infinite number of topics. Your presence in any particular place connects you to the corresponding topic. You can read messages posted to that topic by anyone who’s been there before. If you send a message to the topic, it joins the others there and becomes available to anyone who visits in the future. Of course you don’t have to be physically present to read all the messages posted to a topic. You could look up a topic by its coordinates, and read the topic’s messages from anywhere at any time.

This was a beautiful example of one of the seven ways to think like the web:

6. Participate in pub/sub networks as both a publisher and a subscriber

What the New Scientist described in 2002 may have been an early incarnation of Dodgeball, which begat Foursquare: a pub/sub network organized around a dynamic set of topics. Anyone can create a topic, post messages to a topic, and read messages posted to a topic. The game itself holds no attraction for me. I have never claimed to be the mayor of a bar or an ice cream shop, and I never will. But the pub/sub principle that Foursquare embodies is profoundly important. Wikipedia says there were six million registered Foursquare users in December 2010. I recently registered myself and was given a number just north of 8 million. I’m happy to know that something like that many people have experienced Foursquare’s mode of pub/sub networking. My hope is that as more folks see the underlying communication pattern through the lens of Foursquare, as well as through the lenses of blogging, Twitter, Facebook, and other services, more will grasp the essence of the pattern and find other ways to apply it.

High on the list of other possible ways, for me, is internal corporate communication. In many companies large and small, the dominant paradigm, interpersonal messaging, fails. Sue, who is assigned to Project A, which is hosted on Site X, reaches a milestone and makes a public release. Frank, also assigned to Project A, writes public blog post P in support of the release. Time passes. Sue leaves project A, Roger joins. Also Frank leaves, and Tom joins. Roger rehosts Project A on Site Y, and makes a new release that invalidates Tom’s blog post. How can Tom (who inherited the blog from Frank) know that he needs to update it based on new work by Roger (who inherited Project A from Sue)?

He probably can’t. To understand why not, let’s start with this diagram:

The solid blue arrows are messages that flow among the four human players in this drama: Sue, Roger, Frank, and Tom. Those messages refer to four non-human players, which I’ll call topics: Project A, Blog Post P, Site X, Site Y. The next diagram shows some of those references as dotted black arrows:

Any message can mention one or more topics. If I showed all the possible references the diagram would be very cluttered, so I won’t, but just imagine black dotted lines from every message path to every topic. Do you see what’s still missing? Two critical things:

1. There are no links connecting people to topics.

2. There are no links connecting topics to other topics.

Now let’s redraw the diagram for an environment where there isn’t just person-to-person messaging, but also person-to-topic, topic-to-person, and topic-to-topic. At this point Sue has reached her Project A milestone, made a public release to Site X, and notified Frank, who has written it up as Blog Post P.

In the previous diagram I said the dotted black arrows were references. When Sue wrote to Frank to notify him that she’d reached the Project A milestone, she mentioned the name, Project A, in the subject of her email, or in the body, or both. She also mentioned the name Site X, and its associated URL. When Frank wrote back to Sue to show her Blog Post P, he mentioned its name and associated URL.

In this new diagram, though, I mean something different by the black dotted arrows. Now they are message paths. When Sue reached her project A milestone, she sent a message to a person, Frank, alerting him to that fact. That same message also went to a topic, Project A. Frank could then subscribe to the Project A topic, and find out about things that Sue, or others connected to Project A, knew but didn’t bother to tell him.

If there’s little message traffic flowing through the Project A topic, Frank won’t mind watching it. If the topic gets noisy, he can unsubscribe. But the topic still exists; people who know about it can read its messages; people who don’t know about it can find the topic and its messages by searching. It should go without saying, but this is crucial: topics must be discoverable and searchable.

Frank, meanwhile, by creating Blog Post P, has also created a new topic. Part of what happens next is already common. It’s a blog, therefore it publishes a feed to which any interested party can subscribe. Less common but still not unheard of, in the current state of play, is for the blog post to not only publish feeds (e.g., a feed of blog posts, also per-post feeds of comments) but also subscribe to feeds. So Frank might configure Blog Post P to receive a feed from Project A. When Roger reaches the next Project A milestone, he — or, preferably, his deployment software — posts a message to that effect to the Project A topic. Blog Post P, as a subscriber to the topic, will receive and can optionally republish the message.

Sue, meanwhile, by hosting her Project A release on Site X, has created (or joined) the Site X topic. When she posts the release to Site X, she links it to Project A. The link, in this case, is bidirectional. When Project A reaches a milestone, Site X is notified. And when Site X schedules downtime for an upgrade, Project A is notified.

Now let’s wind the clock forward. Frank and Sue have both left Project A. Frank no longer cares about Blog Post P, and Sue no longer cares about Project A. Roger has inherited Sue’s development role, and Tom has inherited Roger’s public relations role. At this point, however, Roger hasn’t yet reached the next milestone, and so there’s been no need for Tom to revisit Blog Post P. Here’s the picture:

This is what can’t happen in organizations that rely solely on interpersonal messaging. The system retains traces of the connections among Project A, Site X, and Blog Post P. When Roger takes over for Sue, he starts sending messages to Project A, which in turn notifies Blog Post P and Site X. When Roger moves Project A from Site X to Site Y, he redirects that link for all subscribers to the Project A topic. When Roger reaches the next Project A milestone, his message to that effect reaches anyone now subscribed to Site Y. It also reaches Tom, who subscribed to Project A when he took over from Frank.

In theory everyone talks to everyone and everything gets taken care of. In practice, as we know, not so much. Interpersonal messaging alone can’t create a resilient and discoverable web of connections. That’s why interpersonal messaging must be embedded in a pub/sub network where messages flow person-to-person, person-to-topic, topic-to-person, and topic-to-topic.

There isn’t, and never will be, a single product or service that implements this alternative architecture of communication. Many aspects of it are already available to us. You can do a lot of this stuff with internal blogging and social bookmarking, or with an email system, or with a combination of these modes, if people are thoughtful about naming conventions and if the topic archives are discoverable and searchable. But the model is abstract. To make it concrete we’ll need systems that help us use all of our existing communication tools — and all the new ones in the pipeline — to enact this pattern of communication.

In a couple of earlier entries I’ve explored how WolframAlpha can inform public discourse when it involves energy literacy. Last night, when Clemens Vasters tweeted the JAIF (Japan Atomic Industrial Forum) reactor status update, WolframAlpha again showed how helpful it can be. According to the JAIF document, the radiation dose at the border of the power station was most recently reported to be 642 microsieverts/hr. Well, what does that mean?

Q: 642 microsieverts/hr

In the US press, though, the units are typically rems or millirems. So:

Q: 642 microsieverts in millirems

That’s at the border of the power station. What about inside? According to another source, BraveNewClimate, “At 8:47AM on March 16, a radiation level of 300 milli sievert per hour was recorded between the exteriors of the secondary containment buildings of Unit 2 reactor and Unit 3 reactor of Fukushima Daiichi Nuclear Power Station.”

Q: 300 millisieverts

I wish I could flow all the Fukushima news through a WolframAlpha filter that would provide these and other comparisons. More importantly, I wish that the US media would flow their unhelpful coverage (/via @cgerrish) through that filter because Nuclear Boy is doing a better job than Anderson Cooper and Soledad O’Brien.

I just heard a segment of Living on Earth in which Kevin Doran described the privacy concerns raised by smart meters that read the energy signatures of your appliances and report back to the grid. For example, an insurance company might see that you routinely come home and turn on lights at 2AM when the bars close.

Fair enough. But these kinds of stories always seem to end like this:

And as policy makers figure out how to approach this entire new terrain, they need to be cognizant of the fact that there are serious privacy concerns out there, and much of the legislation and statutes that deal with privacy have no idea what to do when it comes to the smart grid because this is so new.

Yes, the smart meter will introduce a new capability, and yes, it will raise issues. But that doesn’t mean that we have to start from scratch every time with “no idea what do to” about those issues. There are core principles; they don’t change; we can use them to navigate the terrain of the new.

There’s no mystery about how this should work. My smart meter will relay data that I own to a service in the cloud that I control. I’ll tell my service who can access my data, and on what terms. We don’t need new rules for every new scenario. We just need one basic rule about data ownership. And then, of course, a data services ecosystem in which we can apply the rule. That’s the goal, and this kind of reporting isn’t helping us get there.

I wish mainstream media’s tech coverage would stop being dazzled by shiny new things and start helping our society learn to think like the web.

Last night I tuned into an interview with a 36 year old @margaretatwood and heard this:

If you don’t own the ground you stand on, you don’t own your right to have a country — you’ve gotten rid of it, you’ve sold it.

This morning, Gardner Campbell’s tweet stream showed me what Kindle public notes look like on Twitter and on Amazon.

As we discussed the other day (Fear not, book lovers. The future of marginalia is bright!) the elephant in the room is the integrity of our digital selves. Gardner’s notes are capillaries that belong to his lifestream, or should, but instead they belong to Twitter, or to Amazon, or in this case to a hybrid of both. We really need to fix that.

We know what happens after Yahoo! acquires you. But what happens before? Here’s some data:

Maybe I should have been a less enthusiastic early adopter of these services?

A story about marginalia in today’s New York Times, Book Lovers Fear Dim Future for Notes in the Margins, opens with an account of a rare and otherwise undistinguished book that’s valuable only because Mark Twain scribbled in its margins:

Like many readers, Twain was engaging in marginalia, writing comments alongside passages and sometimes giving an author a piece of his mind. It is a rich literary pastime, sometimes regarded as a tool of literary archaeology, but it has an uncertain fate in a digitalized world.

“People will always find a way to annotate electronically,” said G. Thomas Tanselle, a former vice president of the John Simon Guggenheim Memorial Foundation and an adjunct professor of English at Columbia University. “But there is the question of how it is going to be preserved. And that is a problem now facing collections libraries.”

Actually it’s a problem facing everyone, and if we solve it for ourselves we’ll solve it for libraries too. The Times story wanders off into nostalagia without proposing any solution. Here’s my proposal for the next Mark Twain and for all the rest of us too: a network of cloud-based personal data stores.

When Mark Twain v1 wrote his marginalia, he had to commit them to a single physical copy of a book. His notes were available only to him, and even then not very effectively. He couldn’t search for one of his own comments. There was only one way to access it. If he wasn’t where the book was, that path was temporarily blocked. If he lost the book, he lost the comment.

Mark Twain v2 will be a citizen of the web. He will possess, among other habits of highly effective web citizens, the habit of communicating by reference rather than by value. In this case, he’d start by citing the passage he was annotating. Whether he was reading a print or electronic book, the citation would encode some facts: author, title, edition, page number, paragraph number. Writing down all those facts for every marginal note would be onerous, but Mark Twain v2 won’t have to, he’ll use software that automates the drudgery and enables him to write as simply as: “pg 52, para 3: Nonsense! I published Huck Finn…”

The citation refers to, but is not part of, the book to which it refers. That’s one level of indirection, and nowadays it’s a familiar one. We have all created and used URLs that point to pages of books, or even to paragraphs within those pages.

The next level of indirection is less familiar. Mark Twain v1′s note was inscribed in the margin of a particular copy of a particular edition of a book. Mark Twain v2′s note can, at a minimum, refer to every copy of that edition. But ideally it can do better. It can refer to every copy of every edition that contains the referenced passage. That’s hard to achieve today in the realm of conventional books, which don’t afford edition-independent ways to refer to works, or to paragraphs, or to sentences within those works.

But the web shows how it can be done. There are, for example, a variety of ways to refer to a work. There’s no consensus as to which is best, and poor interoperability among the various schemes, but it’s a start.

There’s also a longstanding web tradition of intra-work citation. Back in 2000, I wrote a report called Internet Groupware for Scientific Collaboration. The document uses a technique called Purple Numbers, one of the many spinoffs from Doug Engelbart’s pioneering work, to create an URL for each paragraph. What’s more, each of those Purple Numbers (in my case, actually, they were Green Numbers) linked to a discussion board.

Amazingly that discussion board still survives at QuickTopic, but it was never very useful. That’s because a crucial third kind of indirection was, and remains, missing in action. Ideally the authors of those QuickTopic comments would have committed their words to personal web archives under their control, and then syndicated their comments to QuickTopic. Because we still lack that capability, Mark Twain v2 can at best exploit the principle of indirection for his own purposes. He can write marginalia that refers to works, and he can store his marginalia in the cloud for anywhere/anytime access and for safekeeping.

For Mark Twain v1 that might have been plenty good enough. He could rant privately, knowing that his marginalia — like his autobiography — would be available to scholars and to the public after his death. But Mark Twain v2 expects more. Like his namesake, he wants to control access to his lifestream and assure its continuity. But he also wants selected bits of that lifestream to influence the world now instead of later, on terms he defines. A comment that refers to pg 52, para 3 in a work can be declared private or public. If public, it can syndicate to any web context that refers to pg 52, para 3 of that work, while remaining tethered to the authoritative source: a public facet of Mark Twain v2′s lifestream.

A lot of pieces need to fall into place to enable this scenario. Happily they are, for many reasons, the right pieces.

An anti-pattern, Wikipedia says, is a design pattern “that may be commonly used but is ineffective and/or counterproductive in practice.” The example most familiar to me is the password anti-pattern which describes sites that use your credentials to impersonate you on other sites. My work on the elmcity project has surfaced a few other anti-patterns, including one I call the submit-your-event anti-pattern. On websites with events pages, every invitation to send in event information by email, or to type it into a web form, is an example of this anti-pattern. Why? It breaks most of the seven habits of highly effective web citizens, especially #1 (“Be the authoritative source for your own data”) and #2 (“Pass by reference not by value”).

The pattern I recommend invites sources of event data to submit it by publishing feeds, and aggregators of event data to acquire it by subscribing to feeds. Since the legacy forms-based method has such deep roots, you might want to keep it going while offering the feed-based method as a preferred alternative, like so:

The submit-your-event anti-pattern is being rolled out in a major way at Patch, AOL’s foray into local news. On every events pages, like this one for East Providence, RI, Patch invites you to sign up, log in, and populate its database. I had noticed this in my travels through the eventsphere, and thought of it again when I read Ken Auletta’s (paywalled) New Yorker article this week, which profiles Tim Armstrong’s mission to save AOL by building a network of local news hubs. Here was the inspiration for the Patch events service:

While he was at Google, Armstrong had his revelation about local news. One Saturday morning in 2007, he and his children were driving home from a bagel store half a mile from their home, in Riverside, Connecticut. At a stoplight, they pulled over to look at the hand-lettered signs that residents had stuck in the grass to advertise local events. There was no online listing of events in Riverside, and the Greenwich Time lacked a calendar. Armstrong called the newspaper and introduced himself as a resident and told an editor that the paper was missing out on a terrific business opportunity.

“We really don’t need any help. We have a fine business,” the editor told him, before saying thanks and hanging up.

This was crazy, Armstrong recalls thinking. He lived in one of the wealthiest towns in America, yet he had to drive to a stoplight to find out what to do with his family.

event posters, keene, spring 2005

I had a similar revelation back in 2005, when I walked around Keene, photographed all the event posters on shop windows and kiosks, and compared the results to listings in print and online. The posters collectively told much more about goings-on in town than did any of the listings, or indeed all of the listings combined.

A few years later, I came to a very different conclusion than the one Armstrong reached. The kiosk-and-shop-window system was outperforming the web because it was, in one crucial way, more weblike than any existing web-based events system. You don’t have to ask permission to post flyers, you just post them where they can be seen by everyone.

Imagine if the web used a submit-your-web-page anti-pattern. To post a page to the web you’d visit a site, register, log in, fill in a form, hit submit, and wait for approval. Well of course you can’t imagine that, because there wouldn’t be a web if things had to work that way.

To work as well as the web, the eventsphere has to work like the web. If you have events to promote, you post them to your own site. That’s the authoritative source. The information is displayed there as text for people to read, but is also available as data for people and machines to syndicate. Local media hubs don’t get to be the exclusive owners of a database of events because there is no database of events, there are only feeds from authoritative sources. They do, if they’re savvy web thinkers, get to play a preeminent role in the eventsphere by inviting contributors to light up their feeds, and by offering rules and tools that help contributors manage them.

Back in 2000, the patterns, principles, and best practices for building web information systems were mostly anecdotal and folkloric. Roy Fielding’s dissertation on the web’s deep architecture provided a formal definition that we’ve been digesting ever since. In his introduction he wrote that the web is “an Internet-scale distributed hypermedia system” that aims to “interconnect information networks across organizational boundaries.” His thesis helped us recognize and apply such principles as universal naming, linking, loose coupling, and disciplined resource design. These are not only engineering concerns. Nowadays they matter to everyone. Why? Because the web is a hybrid information system co-created by people and machines. Sometimes computers publish our data for us, and sometimes we publish it directly. Sometimes machines subscribe to what machines and people publish, sometimes people do.

Given the web’s hybrid nature, how to can we teach people to make best use of this distributed hypermedia system? That’s what I’ve been trying to do, in one way or another, for many years. It’s been a challenge to label and describe the principles I want people to learn and apply. I’ve used the terms computational thinking, Fourth R principles, and most recently Mark Surman’s evocative thinking like the web.

Back in October, at the Traction Software users’conference, I led a discussion on the theme of observable work in which we brainstormed a list of some principles that people apply when they work well together online. It’s the same list that emerges when I talk about computational thinking, or Fourth R principles, or thinking like the web. Here’s an edited version of the list we put up on the easel that day:

1. Be the authoritative source for your own data

2. Pass by reference not by value

3. Know the difference between structured and unstructured data

4. Create and adopt disciplined naming conventions

5. Push your data to the widest appropriate scope

6. Participate in pub/sub networks as both a publisher and a subscriber

7. Reuse components and services

1. Be the authoritative source for your own data

In the elmcity context, that means regarding your own website, blog, or online calendar as the authoritative source. More broadly, it means publishing facts about yourself, or your organization, to a place on the web that you control, and that is bound in some way to your identity.

Why?

To a large and growing extent, your public identity is what the web knows about your ideas, activities, and relationships. When that knowledge isn’t private, your interests are best served by publishing it to online spaces that you control and use for the purpose.

Related

Mastering your own search index, Hosted lifebits

2. Pass by reference rather than by value

In the case of calendar events, you’re passing by value when you send copies of your data to event sites in email, or when you log into an events site and recopy data that you’ve already written down for yourself and published on your own site.

You’re passing by reference when you publish the URL of your calendar feed and invite people and services to subscribe to your feed at that URL.

Other examples include sending somebody a link to an article instead of a copy of the article, or uploading a file to DropBox and sharing the URL.

Why?

Nobody else cares about your data as much as you do. If other people and other systems source your data from a canonical URL that you advertise and control, then they will always get data that’s as timely and accurate as you care to make it.

Also, when you pass by reference you’re enabling reuse (see 7 below). The resources you publish can be recombined, by you and by others, with other resources published by you and by others.

Finally, a canonical URL helps you measure how the web reacts to your data. If the URL is cited elsewhere you can discover those citations, and you can evaluate the context that surrounds them.

Related

The principle of indirection, Hyperlinks matter

3. Know the difference between unstructured and structured data

When you create an events page on your website, and the calendar on that page is an HTML file or a PDF file, you’re posting unstructured data. This is information that people can read and print, and it’s fine for that purpose. But it’s not data that networked computers can process.

When you publish an iCalendar feed in addition to your HTML- or PDF-based calendar, you’re publishing data that machines can work with.

Perhaps the most familiar example is your blog, if you have one. Your blog publishing software creates an HTML page for people to read. But at the same time it creates an RSS or Atom feed that enables feedreaders, or blog aggregation services, to automatically collect your entries and merge them with entries from other blogs.

Why?

When you publish an iCalendar feed in addition to your HTML- or PDF-based calendar, you’re publishing data that machines can work with.

The web is a human/machine hybrid. If you contribute data in formats useful only to people, you sacrifice the network effects that the machines can promote. If you also contribute in formats the machines understand, they can share your stuff amongst themselves, convey it to more people than you can reach through word-of-mouth human networks, and enable hybrid human/machine intelligence to work with it.

Related

The laws of information chemistry, Developing intuitions about data

4. Create and adopt disciplined naming conventions

When people publish calendars into elmcity hubs, they can assign unique and meaningful URLs and/or tags to each event they publish. And they can collaborate with curators of hubs to use tag vocabularies that define virtual collections of events.

The same strategies work in all web contexts. Most familiar is the first order of business at every conference attended by web thinkers: “The tag for this conference is ______.” When people agree to use common names in shared data spaces, effects like aggregation, routing, and targeted search require no special software.

Why?

The web’s supply of unique names (e.g., URLs, tags) is infinite. The namespace that you can control, by choosing URLs and tags for the things you post, is smaller but still infinite. Web thinkers use thoughtful, rigorous naming conventions to manage their own personal information and, at the same time, to enable network effects in shared data spaces.

Related

Heds, deks, and ledes, The power of informal contracts, Permalinks and hashtags for city council agenda items, Scribbling in the margins of iCalendar

5. Push your data to the widest appropriate scope

When you speak in electronic spaces you can address audiences at varying scopes. An email message addresses one or several people; a blog post on a company intranet can address the whole company; a blog post on the public web can address the whole world. Web thinkers know that keystrokes invested to capture and transmit knowledge will pay the highest dividends when routed to the widest appropriate scope.

The elmcity example: a public calendar of events can be managed in what is notionally a personal calendar application, say, Google Calendar or Outlook, but one that can post data to a public URL.

For bloggers, this principle governs the choice to explain what you think, learn, and do on your public blog (when appropriate) rather than in private communication.

Why?

Unless confidentiality precludes the choice, web thinkers prefer shared data spaces to private ones because they enable directed or serendipitous discovery and ad-hoc collaboration.

Related

Too busy to blog? Count your keystrokes

6. Participate in pub/sub networks as both a publisher and a subscriber

Our everyday calendar programs are, in blog parlance, both feed publishers and feed readers. Individuals and organizations can publish their own feeds to the web of calendar data while at the same time subscribing to others’feeds. On a larger scale, an elmcity hub subscribes to a set of feeds, and in turn publishes a feed to which other individuals (or hubs) can subscribe.

Why?

The blog ecosystem is the best example of pub/sub syndication among heterogeneous endpoints through intermediary services. Similar effects can happen in social media, and they happen in ways that people find easier to understand, but they happen within silos: Facebook, Twitter. Web thinkers know that standard protocols and formats enable syndication that crosses silos and supports the most open kinds of collaboration.

Related

Personal data stores and pub/sub networks

7. Reuse components and services

In the elmcity context, calendar programs are used in several complementary ways. They combine personal information management (e.g., keeping track of your own organization’s public calendar) with public information management (e.g., publishing the calendar).

In another sense they serve the needs of humans who read those calendars on the web while also supporting mechanical services (like elmcity) that subscribe to and syndicate the calendars.

In general, a reusable web resource is:

1. Effectively named
2. Properly structured
3. Densely interconnected (linked) both within and beyond itself
4. Appropriately scoped

Why?

The web’s “small pieces loosely joined” architecture echoes what in another era we called the Unix philosophy. Web thinkers design reusable parts, and also reuse such parts where possible, because they know that the web both embodies and rewards this strategy.

Related

How will the elmcity service scale? Like the web!, How to manage private and public calendars together

In 2006, operating quietly behind the scenes, Dan Thomas and Suzanne Peck alerted me to what would become the municipal open data movement we know today. As the ball got rolling, though, I felt that something was missing. It’s great that citizens are now learning to expect access to municipal data, and to expect useful online services to flow from such access. But citizens are providers of data too. We need to expect one another to provide the data for which we are individually and collectively authoritative.

One of my favorite taglines for the municipal open data movement is Mark Surman‘s evocative phrase: cities that think like the web. The elmcity project aims to help cities do that. Next week I’ll be in Toronto for a series of meetings and also a public talk. I want to suggest that in cities that think like the web, citizens understand and apply “fourth R” principles. They know something about how data can be structured for humans to read versus for computers to process. They recognize that pub/sub syndication is a good way to merge their own data into the public ecosystem. They take responsibility for publishing their own data in useful ways, and they expect their fellow citizens to do the same.

If you’re a Torontonian who’s interested in these ideas, we’ll be discussing them on Tuesday afternoon at the University of Toronto’s Cities Centre (John H. Daniels Faculty of Architecture, Landscape, and Design, 230 College Street, Room 103). And if you know Torontonians who aren’t technical but who care about these ideas, please do alert them. They’re the ones I particularly need to reach.

The elmcity project’s single biggest hurdle continues to be a conceptual one. People mostly lack the intuition that it’s possible — never mind easy and free — to publish data that can syndicate. In response to an earlier item on this topic, Stefano Mazzocchi (Cocoon, Simile, and Google Refine) offered some thoughts which I’m sharing with his permission:

A few weeks ago, my in-laws were visiting. She is a pretty famous book author and we were talking about how technology could bring value to her workflow (she is not flat out an IT luddite but close enough).

She just created her first web site and asked me how she could promote it on search engines (classic newbie SEO question). She was not interested in the mechanics at all, she just wanted more exposure.

We talked about Twitter and about how publishers and authors use it to promote themselves and engage their audiences. She thought all this was very “Hollywood” and not her style at all. But I showed her that you don’t need to use Twitter that way, you can just mine it for your ego network. Then I explained how I set up all sorts of traps around the web, with newsfeeds, and how I use Google Reader to aggregate them all for me.

I showed her right there and then. Searched for her new book name on Twitter, clicked on the RSS feed, did the same on Google blog search, Google news search, and voila, her personal PR aggregation network was born.

She was completely blown away. She didn’t know any of this was even remotely possible, yet, once explained, it make perfect sense. It’s like having personal agents watching everything that goes on and sending you the information. Email versus RSS doesn’t make any difference to her. As long as she has a place to go and check out what others say about her, she’s happy.

My take: no tech-unsavvy person thinks it reasonable to have a personal agent that does, for individuals and for free, what gigantic organizations struggle to do every day.

The fact that Google can search 15 billion pages in milliseconds doesn’t faze them as much. If librarians can do it, so does Google. Big deal.

But personal agents constantly working in the cloud for you? It doesn’t even show up in the realm of possibilities.

If Stefano’s in-law were on Facebook, of course, she’d be getting a sense of what it’s like to have one of those agents in the cloud. Her activity stream would magically be visible to friends, and their reactions to it would magically be visible to her. That’s why I often say, nowadays, that Facebook is a great set of training wheels for the pub/sub network.

But Facebook isn’t, yet, a place where people can learn how to publish data that syndicates beyond Facebook. It’s possible, as I discussed in Heds, deks, and ledes, to post public events on Facebook in a way that can be discovered by an elmcity hub or by some other agent. If you don’t know such agents can and do exist, though, you’ll never stop to think about whether they’re actually finding your data — and if not, how to make sure that they do.

One of the key points embedded in Stefano’s parable is that his in-law didn’t have to do anything special in order to be able to find the web’s reaction to her book. To the extent that her name and the name of her book are out there and indexed, they provide good-enough hooks for search aggregation.

Over time, of course, the efficacy of these searches will decay. I’ve watched this happen with my own name. Years back, my stuff was pretty much the only stuff that a search for Udell would find. (There was even a time when the first Jon on Google was me, not Jon Stewart!) Then my wife began showing up, along with a whole bunch of other Udells. So I tuned my filters to Jon Udell, and they work better for now, but there are other Jon Udells and it’s only a matter of time before that namespace gets cluttered too.

In order to reliably find stuff about me, I need filters tuned to aspects of my identity: my domain name, my Twitter handle. Eventually Stefano’s in-law may reach the same conclusion. She may realize that posting to her website is more than a way to share her thoughts with the world. It also enables the world to react to her posts in ways she can, in turn, discover. At that point she may start to see why it’s important to actively colonize parts of the web that are, or can be, bound to aspects of her identity.

The elmcity project, similarly, invites promoters of public events — and communities at large — to colonize the web in ways bound to those individual or group identities. When you produce a calendar feed that flows through an elmcity hub, you’re not just helping to populate that hub. That feed is attached to your own site and, in theory, is directly discoverable there. In practice, though, there are aren’t yet good methods of discovery. We don’t yet have, for iCalendar, an autodiscovery mechanism like the one we have for RSS. That’d be easy enough, as Mark McClaren suggests:

Love it or hate it, iCalendar is the pervasive calendaring format. If we can enable RSS autodiscovery then why not do the same with iCalendar feeds. Adding one line of code would make it easier for people/machines to subscribe to an iCalendar feed.

<link rel="alternate" type="text/calendar"
  title="iCalendar feed for example.com"
  href="calendar.ics" />

It would also be really helpful to be able to bind locations to events in a discoverable way. To that end I’ve recently enhanced the HTML rendering of elmcity hubs. Now they include what Google calls rich snippets, using the RDFa-style markup documented here. The snippets include latitude and longitude coordinates derived in one of two ways:

1. Per-event. There are several ways that an event can show up bearing latitude/longitude values. The vast majority of such events will be those coming from Eventful and Upcoming, both of which services provide lat/lon values via their APIs. There’s also a GEO property defined for iCalendar, and some iCalendar producers use it to geocode events.

2. Per-hub. Although most iCalendar producers don’t use the per-event GEO property, elmcity hubs know their own locations. So events that lack specific lat/lon coordinates inherit the locations of their hubs.

It’s going to be a while yet until folks like Stefano’s book-writing in-law start to realize they can, as Kingsley Idehen nicely puts it, master their own seach indexes. But sooner or later they’ll realize that it’s possible. Likewise, it’ll be a while yet until promoters of public events realize that the event data they push to their websites can not only feed pub/sub networks, but can also feed location-aware search engines. I’m a patient man, though, and I do expect the seeds I’m planting to grow and eventually bear fruit.

Nowadays when people ask if I’ve read a book and I start to answer yes, I have to stop and think. Did I actually read the book? Or did I only hear the author discuss the book on a podcast? This confusion wouldn’t happen if the book were a work of fiction, but I’m mainly drawn to non-fiction and in that realm I’ve noticed a couple of things. First, I seem to absorb the gist of non-fiction books so well from listening to their authors that I sometimes feel as if I’ve read them. Second, I find that when I do read these books I am sometimes disappointed to find that the writing doesn’t compel me in the same way that the speaking did.

I almost hate to mention this effect, because book publishing is a tough business already and doesn’t need more grief. Nor would I want authors to fear audio exposure. But the effect is real, at least for me, and I wonder why. Here are two theories:

1. The rebirth of the oral tradition

Before there was print, we mainly experienced writing as authors’voices. Print expanded the reach of their words but not of their voices. During the 20th century, electronic media expanded the reach of some authors’voices — but only those few who appeared in mainstream media. In the 21st century, though, podcasting has democratized interviews with — and lectures by — authors. Now, for almost any book, you can find one or more podcasts in which the author discusses the work. We have far greater access to the voices of the authors we read and, through their voices, to their personalities. The voices and personalities can be more compelling than the writing.

2. The process of iterative refinement

When you read a book, you access the author’s brain at the moment when the book was just finished. When you listen to an author discuss a book, though, you access his or her brain after it has reflected on the book and processed the world’s reaction to it. That later brain knows more about the themes of the book, and can articulate them better.

Spoken-word audio occupies a small niche within the ecosystem of downloadable audio, so maybe few are noticing this effect. That’s probably a good thing. I like accessing authors’brains through their voices in addition to — but sometimes as a substitute for — their written words. That kind of substitution, if more widely practiced, would be disruptive.

A non-profit organization chartered to promote arts and culture within its community will be curating a new elmcity hub. The curator plans to invite dozens of member organizations to contribute to the hub — that is, to manage their public schedules using calendar applications, and to convey the URLs of their calendar feeds to the hub. It’s critical that these member organizations will be able to easily accomplish this task. I’ve been pointing to a series of how-to articles but it was time to make the instructions clearer and simpler. It was also time to revisit how to use screencasting to explain step-by-step procedures.

I started with the two web applications that make publishing calendar feeds as simple as it can be: Google Calendar and Hotmail Calendar. Here’s the new and improved explanation of How to publish a calendar feed from Google Calendar or Hotmail Calendar. I’ll soon learn how well this new explanation works for the intended audience. Meanwhile, here are some notes on the techniques I discovered.

Using sparkcasts

My earlier explanations didn’t use screencasts, they were just textual narrations with embedded static screenshots. That’s an appropriate way to explain step-by-step procedures. Screencasts can be overkill.

But it is nice to be able to show motion and sequential flow. I wondered: if sparklines are “intense word-sized graphics” that appear inline with text, is there a similarly lightweight way to use screencasts? Well, that’s how this explanation works. The steps are all written as text. Each step comes with a Watch! link that opens a quick little inline screencast.

Using Firebug to prepare screencasts

As I mentioned the other day, Firebug is an amazing tool for making mockups based on live web pages. And sometimes you can do more than just take screenshots of those mockups. Sometimes you can interact with them.

I used that method to temporarily alter both Google Calendar and Hotmail Calendar for these sparkcasts. In my case, both include several private calendars that were irrelevant to my explanation. Using Firebug I was able to remove them. I also made the address shown in Gmail look like the generic YourAccountName@gmail.com instead of judell@gmail.com. If subsequent interaction with the page doesn’t trigger a refresh of these altered elements, the changes will persist. Handy!

Using Camtasia to make animated GIFs

A movie-style screencast is a fairly heavy object, and it would be cumbersome to embed lots of these on a page. Animated GIFs are lighter. And it turns out that Camtasia can produce that format. Nice!

Using jQuery to reload animated GIFs

On the first try, I made each sparkcast loop forever, and used jQuery to show only the current one. But that wasn’t ideal. What I really wanted was for each sparkcast to play once when shown, then stop and offer a Replay link. If you view the source of this page, you’ll see the solution I came up with. It turns out that when you dynamically reload an animated GIF, you need to decorate its URL in a unique way; otherwise the browser won’t replay it.

Using Image Magick to control the looping of animated GIFs

Once I figured out how to replay a play-once GIF, I had to make all the GIFs stop looping. I’m sure that PhotoShop and others of its ilk can do that, but I like Image Magick’s ability to automate repetitive tasks. The Image Magick incantation to make an animated GIF play once goes like this:

mogrify -loop 1 howto-01.gif

Now that I’ve written this down, I can forget it until I need it again!

Eric von Hippel’s book Democratizing Innovation rang all sorts of bells for me back in 2005. I cited it in a couple of InfoWorld columns and continue to cite it in blog posts. Here’s how I originally summarized the book:

MIT’s Eric von Hippel notes that users of products and services — and by users he means both individuals and companies — often innovate on their own rather than relying on manufacturers to do it for them. And not just in the realm of IT; a survey of employees in 74 pipe-hanger installation companies found that 36 percent developed or modified pipe-hanger hardware for their own use.

There’s no mystery why this should be. In general, we all share the same needs, but specific requirements vary in ways that motivate a shift from mass production to mass customization. The question is how to do mass customization economically.

Von Hippel advances the notion of user innovation toolkits. The Apache Web server, with its modular architecture, is an example of such a toolkit. In the hands of skilled programmers, Apache can be, and often is, tailored to specific needs. When such customizations are shared, other users benefit. But so do Apache’s developers, who, by observing what’s done with the toolkit, can more intelligently evolve the core product.

I’m always on the lookout for user innovation toolkits, and what brings me back to the topic today is this analysis of the design of Google’s and Bing’s maps (/via @gnat). It includes a stunning animated gif that flips between the Google and Bing renderings of the northeast U.S.

Justin O’Bierne’s question was: “Why Do Google Maps City Labels Seem Much More “Readable” Than Those of Its Competitors?” It’s a fascinating and, in my non-expert judgement, a very plausible analysis. When I watch the two sample images in rapid alternation, though, I’m struck by the way in which each seems more readable than the other in certain ways. I can imagine a mixing board with knobs for a bunch of textual, graphic, and layout parameters. By twiddling the knobs I could combine what I like best about each of these two samples. Of course I’m not a designer. If you are a designer, you’ll say that I’m being naive, that design isn’t knob-twiddling, it’s a holistic exercise. And you’ll be right. When everyone can twiddle all the knobs, disasters can ensue. I’ve created a few of my own.

Still, if we can democratize the knob-twiddling, we should. User innovation toolkits can’t guarantee good outcomes, but they can open up the design process to experiments that might sometimes work.

In that vein, I want to draw attention to an interesting use of Firebug. If you’re a web developer you know about this excellent tool. If you’re not you probably don’t, but you might want to remember what I’m about to say. It turns out that Firebug can enable an armchair web designer, perhaps with minimal knowledge of HTML and CSS, to prototype in-situ alternate versions of live web pages.

I discovered this because I wanted to be able to show how the elmcity event viewing widget might look as a replacement for the existing widget on the events page of various web sites. I formerly did this kind of thing in two ways. One was to drop an image of the new element onto an image of the existing page. The other was to save the page, edit its HTML and CSS, and snapshot the result. For all sorts of reasons, both of these methods were flawed and, more importantly, inacessible to anybody without a ton of web development experience.

Then I realized that Firebug enables a better way. Clearly I’m late to the party. I’m sure most web developers have known about this technique for a long. But I’ll bet a lot of people who aren’t web developers would want to use it from time to time too, if they knew how. So, here’s the drill.

Let’s say that I want to show the Keene Sentinel how the elmcity widget would appear on the Sentinel’s calendar page. Here’s the existing page:

I position the cursor on the calendar element, and inspect it:

Now I can see that the element I want to replace is <div id="calendar">. So I click its parent, <div id="inside-page">, inspect it, click Firebug’s Edit button, and remove everything inside that tag:

Edit button??!! Who knew? Many of you, I guess, but not me until recently. Anyway, to complete the task I follow the instructions for how to stick an elmcity viewer widget into a page:

Done! Here’s what the Keene Sentinel’s calendar page would look like if it used the elmcity widget:

Firebug isn’t just a great tool for developers. It’s also a user innovation toolkit that can enable anyone to try out an alternate version of a live web page.

Here’s the blurb for a lunchtime talk I’m giving next Tuesday, December 7, 12:30 pm, at Harvard’s Berkman Center. Update: Slides here.

The elmcity project invites everyone who publishes community calendar events to:

• Realize that event data published in a structured format, unlike data published as HTML or PDF, can be routed through pub/sub syndication networks.
• Make public calendars available in the appropriate structured format: iCalendar (RFC 5545), the venerable Internet standard supported by all major calendar applications and services.
• Recognize that iCalendar is the RSS of calendars. It can enable a calendar-sphere in which, as in the blogosphere, everyone can publish their own feeds and also subscribe to feeds from other people or from network services.
• Help build the data web by owning the parts of it for which we ourselves are the authoritative sources.

The elmcity project delivers enabling technical infrastructure for this new approach to the community calendar. The project’s calendar syndication service is free; it runs open source code on the Microsoft Azure platform; it provides all of its syndicated data in open formats.

The real challenge isn’t technical, though, it’s conceptual. Most people don’t know how they could (or why they should) be the authoritative publishers of their own data. Missing concepts include:

• The pub/sub communication pattern
• Indirection (“pass-by-reference” vs “pass-by-value”)
• Structured versus unstructured data
• Data provenance
• Service composition

Along with reading, writing, and arithmetic, these Fourth R principles will empower an informed and engaged 21st-century citizenry. As Jeannette Wing argues in her computational thinking manifesto, computer and information scientists are no longer the only ones who need to understand and apply these principles. Now we all do.

Drawing from the experience of the elmcity case study, this talk will explore what these Fourth R principles are, why they’re hard for most people to understand, how we can teach them, and why we should.

When a curator starts up an elmcity hub, one of the outputs is an HTML view of the hub’s aggregated events. It’s just a scrolling list of H3 elements, augmented with a datepicker. In its first incarnation the datepicker was based on the Yahoo datepicker widget. Later I switched to the jQuery UI datepicker, and used it to:

• Highlight the current date

• Scroll the page to a selected date

• Adjust the current date when the users scrolls the page

There was a lot more mileage I could have been getting out of the jQuery widget, though. This weekend I added the following behaviors:

• Display only those sources — from the list [Eventful,Upcoming,EventBrite,Facebook] — that the curator has included

• Display the count of iCalendar feeds the curator has included

• Use hanging indentation to make the list of events more easily scannable

• Enhance the highlighting of the current date

• Dim the days that have no events

• Scroll the page to a selected month when the user clicks the widget’s buttons

• Disable the widget’s buttons at the beginning and end of the range of available dates

• Search within the page for the first occurrence of an event matching the search term

One of the places where you can see the new widget in action is at Berkeleyside, the independent local news site for Berkeley, California.

Of course there are many ways to skin the cat. At another California-based hyperlocal site, Menlo Park’s InMenlo, the aggregated iCalendar feed (another of the elmcity service’s outputs) is displayed in an instance of Google Calendar. Note that in this scenario the only view that makes sense is the list (agenda) view, since a hub with any kind of flow will quickly overwhelm the other views.

Not long ago I wouldn’t have thought it possible to make a credible alternative to Google Calendar’s display widget. But with libraries like jQuery UI, along with tools like Firebug and the formidable debugger that’s available in the IE9 preview, I’m able to stretch my modest skills farther than I once could. I’m fairly happy with the current version of the viewer, and now that I’ve started to really get the hang of jQuery I’m looking forward to improving it.

I’d like to thank Caleb Clark for recording and posting a video of the talk I gave last month at the Marlboro College Graduate School. I watched it the other night and I think it’s my best explanation of a cluster of things I’ve been thinking about and working toward for a long time. The list includes:

• the local web
• LibraryLookup
• webscale identifiers
• REST
• public data
• loosely-coupled cloud services
• lightweight service composition
• structure and transformation of data
• the elmcity project
• the pub/sub pattern
• feed syndication
• personally authoritative data publishing
• social and decentralized information management

When I look at that list, and realize that I’m always trying to do (and describe) all of these things at the same time — because they’re all deeply intertwingled — I can see why it’s been so hard to tell the story. Apparently, given an hour, I can now tell it reasonably well. But I’ll rarely get that hour. So I also need to condense it into a five-minute Common Craft-style summary. A hard challenge, but a good one!

Until a few days ago I never owned, carried, or used a smartphone. That made me an anomaly not only in geek circles but, increasingly, among civilians too. I had always been the pioneer adopter. Now I found myself at dinner parties watching friends do the kinds of things that they always used to watch me do: Drift away from the conversation, engage with unseen interlocutors, jack into the planetary dataspace.

The experiment was less inconvenient for me that it would be for many others. I work from home, I’m rarely offline, and I could use my feature phone’s primitive data services in a pinch. Still, why? Because, as William Powers says in Hamlet’s BlackBerry, “The air is full of people.”

Someone you know has just seen a great movie. Someone else had an idle thought. There’s been a suicide bombing in South Asia. Stocks soared today. Pop star has a painful secret. Someone has a new opinion. Please support this worthy cause. He needs that report from you — where is it? Someone wants you to join the discussion…

The subtitle of Powers’book is A practical philosophy for building a good life in the digital age. He wants us to question our “digital maximalism” — that is, our uncritical embrace of connectivity for its own sake. But he frames the question using a series of historical examples. Technology, he argues, has always played a complex dual role as a mediator between our inner lives and the crowd.

In the first example, from Plato’s Phaedrus, Socrates and Phaedrus leave the connectivity-amplifying city for a walk in the countryside, so they can enjoy a deep private discussion about a lecture that Lysias had given. But first Socrates wants Phaedrus to recite the lecture, and expects him to do so from memory. Phaedrus says he can’t, and produces a written copy. This was “the very latest communications technology” — one that Socrates was wary of, but that here enables an experience that combines withdrawal from the hive and engagement with it.

I’ve been reflecting on continuous partial attention, and the shallowing effects of cyber-augmentation, for a while now. It’s why I took a break from this blog, put my podcast on pause, and sat out the early phase of the smartphone era. But it was inevitable that I’d get a smartphone someday, and when Microsoft made an offer I couldn’t refuse, I did.

As it turned out, this past Friday was the day. On Saturday, driving down to Boston with the family for an outing, I rode shotgun so I could explore the new thing. But I was determined to use it in a balanced and appropriate way. Since we were headed to Cambridge, and since there’s an elmcity hub for Cambridge, I checked it and found out about the Horns and Antlers exhibit at Harvard’s Peabody Museum. That’s right up Luann’s alley so we decided to go.

Then, feeling slightly conflicted, I dipped into the Twitter stream and read this:

@gardnercampbell: Just found & bought new poems by Gjertrud Schnackenberg. Harvard Bookstore makes my day.

Really? One of my favorite people is visiting Cambridge the same day? Shades of manufactured serendipity. And lo, Gardner and I were able to continue a dialogue we’ve been having for years, but rarely face to face.

So what do I think of Windows Phone 7? I love it and I fear it. Now admittedly, I would love an iPhone or an Android too. So if you know these devices you’ll need to look elsewhere for a comparative review.

Then there’s the fear. During the relatively few periods when I could have been connected to the crowded cloud but wasn’t, I’ve reflected on my own uncritical embrace of digital maximalism. So I do worry about carrying the crowd in my pocket. But I hope I’ll figure out how to strike the proper balance. One thing I’m pretty sure of: you won’t find me electing myself mayor of a coffee shop.

Sometime in the latter 1990s I was looking for a passage in a book that I owned. It was a revelation to discover that I could find the passage online more easily than I could by first locating my copy of the book, then scanning it and using its index. I’ve since re-enacted that scenario many times, most recently the other day when I was looking for the Diane Deutsch quotation about perfect pitch that appears in Oliver Sacks’Musicophilia. In this case I had a library copy of the book on my desk. The text I was looking for is on page 125 of the library’s edition. As I like to do now and then, I made notes on the search strategy that got me to that page.

What I remembered of the passage was the analogy between pitch discrimination and color discrimination, so I began by searching the book using, somewhat arbitrarily, Google Books. My search term was simply color. The outcome of this naive attempt was both lucky and unlucky. Luckily it produced the most memorable part of the passage:

Suppose you showed someone a red object and asked him to name the color … Then you juxtaposed a blue object and named its color, and he responded, “OK,

Unluckily there was no preview available for the page. And the number of the found page was given as 134, which didn’t match the library edition I had on my desk. So I switched to Amazon. But the trip through Google Books was not useless. I came away with a much more discriminating phrase with which to search Amazon: red object.

Armed with that phrase, I found the page on Amazon right away, and the preview was available. But it wasn’t fully available: it ended in the middle of the passage I wanted. And again the page was given as 134, which differed from my edition.

Now, though, I had a partial page preview that showed me the layout of the page I was looking for. It was distinguished by a large indented block quote. I also had rough idea of where to look in the book: somewhere near page 134. Armed with these inputs I was able to scan the library book and zero in on page 125.

We don’t often enough name or describe the knowledge, the skills, and the techniques that enable successful search. To the extent that we do, we tend to suggest that there’s a best search engine, or a best search strategy, but the real story is subtler. Often, as in this case, the theme of the story is a pipeline of components. Here’s an illustration of the pipeline:

The mental model that drives this pipeline includes these assumptions:

• There are multiple components. In this case: Google Books, Amazon, and the library book.

• The components are differently searchable. Google Books and Amazon provide fulltext search; the book’s affordances are page-scanning and an index.

• Search results are differently viewable. Google Books and Amazon may or may not provide previews; the book in hand is fully viewable.

• The searchable components yield varying results depending on both input terms and available previews.

• It’s possible, maybe likely, that no single component will lead to the desired result

• A partial result from one search component can be piped into another search component.

I use the same approach when I search the web using Google and Bing in parallel. We have a cornucopia of tools at our disposal. We don’t expect to use the same screwdriver for every task; tools vary in their affordances and uses; we keep an evolving collection in our kits and combine them in novel ways to meet evolving challenges. To speak of a best search engine is as meaningless as to speak of a best screwdriver. When we teach “computer literacy” we need to develop the intuition that there’s no best information tool, but that there is a best model for using these tools.

I thought I’d read everything by Oliver Sacks but he is prolific and I’d fallen behind. So I had to catch up with Musicophilia before I proceed to The Mind’s Eye. One of the themes of Musicophilia is brainworms: catchy tunes that you can’t get out of your head. Another kind of brainworm, for me, is the phrase or quotation that sticks in my head after I finish a book. My Musicophilia brainworm is a quote from Diane Deutsch about perfect pitch, which is another major theme of the book:

To give you a sense of how strange a lack of absolute pitch appears to those of us who have it, take color naming as an analogy. Suppose you showed someone a red object and asked him to name the color. And suppose he answered: “I can recognize the color, and I can discriminate it from other colors, but I just can’t name it.” Then you juxtaposed a blue object and named its color, and he responded, “OK, since the second color is blue, the first one must be red.” I believe that most people would find this process rather bizarre. Yet from the perspective of someone with absolute pitch this is precisely how most people name pitches — they evaluate the relationship between the pitch to be named and another pitch whose name they already know.

…

When I hear a musical note and identify its pitch, much more happens than simply placing its pitch on a point (or in a region) along a continuum. Suppose I hear an F-sharp sounded on the piano. I obtain a strong sense of familiarity for “F-sharpness” — like the sense one gets when one recognizes a familiar face. The pitch is bundled in with other attributes of the note — its timber (very importantly), its loudness, and so on. I believe that, at least for some people with absolute pitch, notes are perceived and remembered in a way that is far more concrete than for those who do not possess this faculty.

I don’t have perfect pitch but I’m starting to wonder if I have something like it in the realm of networked information systems. This week’s essay in my Why and how series, entitled Heds, deks, and ledes, is a case in point. The essay recalls chapter 4 of Practical Internet Groupware, which I wrote over a decade ago, and have reformulated in various ways since. To me the principles are so evident that it’s hard to understand why I had to write them down in the first place, never mind continue to restate them over the years. But I do so because I keep realizing that the “F-sharpness” I perceive is not evident to most people. The qualities of this kind of “F-sharpness” include these awarenesses:

• of the layered structure of a package of information

• of which layers are active in different network contexts

• of how layers interconnect

• of how to compose each layer to maximize its visibility and connectivity

I’ve always believed that these are teachable principles, and I’m striving more than ever to find ways to teach them. But what if they aren’t? What if this kind of “F-sharpness” is wired into my brain in a way it can’t be in most brains? It would be a disappointment but also a relief to realize that what I’m trying to teach might not be broadly teachable. I’m still stuck with the brainworm, though.