Jon Udell

In yesterday’s Keene Sentinel guest editorial, YES: Press for more domestic oil, Andrew Morriss consistently misreports oil production figures. Saudi Arabia: 8.1 billion barrels/day; Russia: 10.4 billion; America: 7.9 billion; Iran: 3.7 billion. Those numbers struck me as wrong by orders of magnitude. So I consulted my favorite online scientific fact-checking tool: WolframAlpha.

Q: What is 7.9 billion barrels of oil in dollars? A: $1.05 trillion.

In other words, if we were to convert our daily production to dollars we could pay off the national debt in 15 days. (Real answer: 41 years.)

Q: What is 7.9 billion barrels in gallons? A: 330 billion gallons.

In other words, America’s daily oil production is 4 times the volume of oil transported worldwide in a year. (Real answer: 2.5 times the volume of one of the largest supertankers.)

Q: What is the energy equivalent of 7.9 billion barrels of oil in BTU? A: 45 quadrillion BTU.

In other words, America’s daily oil production is almost half of the total energy consumed by the US in 2001. (Real answer: 1/1000 of that amount.)

What Andrew Morriss meant to write — four times — was million, not billion. Doing that once might be a typo. Doing it four times makes me question his energy literacy.

Now to be fair, most of us — myself included — are not engineers or scientists. We don’t regularly deal with large numbers, and we don’t intuitively grasp relationships among the quantities, prices, and energy content of the various sources that might power our civilization. But we can’t afford to be illiterate (and innumerate) on the topic of energy. The future of the economy, the environment, and of world politics all hinge on our ability to reason about those relationships. Fortunately there is now a tool that can help us all do that more effectively. WolframAlpha isn’t just a boutique search engine. It’s a compendium of scientific knowledge mated to a scientific calculator that understands questions in plain English. And it frames the answers in ways that make sense to everybody. I commend it to Andrew Morriss, to fact-checkers at the Sentinel, and to readers. For more examples of how WolframAlpha can help us reason about energy, see http://delicious.com/judell/energyliteracy.

As we increasingly augment our minds I sometimes pause to reflect on the trade-offs we are making. What powers does the unaugmented mind possess? What do we give up when we outsource our memories to the collective electronic mind? In Dilemma of a Cyborg Carina Chocano writes:

For everything that’s gained by our ability to store and maintain more information than ever before, something is lost that has to do with texture, context and association. The science journalist Joshua Foer, author of “Moonwalking With Einstein: The Art and Science of Remembering Everything,” said in a lecture to the Royal Society of Arts that people once “invested in their memories, they cultivated them. They studiously furnished their minds. They remembered. Today, of course, we’ve got books and computers and smartphones to hold our memories for us. We’ve outsourced our memories to external devices. The result is that we no longer trust our memories. We see every small, forgotten thing as evidence that they’re failing us altogether.” As we store more and more of what makes us us outside of ourselves, he said, “we’ve forgotten how to remember.”

The mnemonic techniques rediscovered in Moonwalking with Einstein were first popularized by Cicero. You bind memories to images, and then you bind the images to a path through the rooms and hallways of a “memory palace.”

Here’s another technique that isn’t so well known. I attribute it to Carlton Fisk by way of a story I heard from the baseball writer Roger Angell. Somewhere in the 2000s, Angell asked Fisk to reflect on what had most altered the game of baseball since his playing days. The salaries? The drugs?

No. The game-changer, Fisk said, was instant replay. His game-winning 1975 home run is one of most-remembered moments in all of sports. The video of that event is one of the most-watched clips. You might think that Carlton Fisk has seen that clip a million times. But in fact, he told Roger Angell, he never watches it. That’s because he doesn’t want to overwrite the original memory, which is his alone, recorded from a point of view that was his alone, with a memory we all share that was recorded by a camera up in the stands.

We can’t do away with instant replay, nor do we want to. But it’s worth remembering how to experience life, even when we know it’s being recorded externally, as if the only cameras are the ones in our heads.

The other day Tim Bray tweeted a Google+ item entitled Geeks and repetitive tasks along with the comment: “Geeks win, eventually.”

Here’s the chart posted on Google+ by Bruno Oliveira:

A couple of things bothered me about this. First, there’s the adversarial tone. The subtext is a favorite geek quotation:

First they ignore you, then they laugh at you, then they fight you, then you win.

That gem is often attributed to Gandhi. Wikiquote disputes that and finds a close variant in Nicholas Klein’s 1918 speech to the Amalgamated Clothing Workers of America. Either way it’s the story of a persistent underdog overcoming an oppressor.

In geek ideology the oppressors are pointy-haired bosses and clueless users. Geeks believe (correctly) that clueless users can’t imagine, never mind implement, automated improvements to repetitive manual chores. The chart divides the world into geeks and non-geeks, and it portrays software-assisted process improvement as a contest that geeks eventually win. This Manichean worldview is unhelpful.

But the chart also fails to capture the reality of repetition and automation in the realm of information systems. Here’s an alternative world view that I choose to imagine and strive to create:

In this view of the world, tasks that involve data manipulation (as so many modern chores do) are undertaken by teams. There is an infinite supply of manual chores. Everybody tackles them. Ideally there is one member of the team I call the toolsmith. Working shoulder to shoulder with the team, the toolsmith spots an opportunity to automate some piece of the work, writes some code, deploys it, observes how it gets used (or doesn’t get used), assesses its impact (or lack of impact), and then iterates on the code. Meanwhile the toolsmith keeps working alongside the team, chipping away at the never-ending and always-evolving list of manual chores, looking for more opportunities to automate, and exploiting them in an incremental and collaborative way.

Software-assisted automation of repetitive work isn’t an event, it’s a process. And if you see it as a contest with winners and losers you are, in my view, doing it wrong.

Before my podcast went AWOL I had been meaning to interview Toby Considine about his efforts to mesh schedules for things with schedules for people, and to define Internet calendaring extensions for that purpose. So when Phil Windley wrote to ask me how I thought calendaring might relate to the personal event networks he wants to bootstrap, I suggested that we ask Toby on an episode of Phil’s Technometria podcast.

Phil, I suspect, is actually one of five identical quintuplets. One teaches at BYU. Another runs a startup called Kynetx. A couple of others take care of family, church, and political matters. Finally there’s the one who does a podcast and also serves as executive producer of ITConversations. I don’t think I could handle being a quintuplet but it’s lucky for all of us that Phil can!

It was a real pleasure to meet Toby and learn about his comprehensive vision for a world in which our buildings, our cars, and our energy grid work with us, appearing as the intelligent agents that science fiction always encouraged us to imagine. One of his favorite examples involves a meeting room that’s scheduled for a 9AM meeting. A really intelligent system wouldn’t start heating or cooling the room at 9AM, it would start sooner. How much sooner? That would depend on the number of people attending.

Orchestrating that kind of dance requires the sort of loosely-coupled event-driven programming at the heart of the Kynetx technology that Phil is creating. Toby is layering that orchestration on top of existing Internet standards: iCalendar (for events), vCard (for resources), LDAP (for directory services). I love that approach for two reasons. First, I’m forever being reminded that we have barely scratched the surface of what might be accomplished with these existing standards if we really put them to use. Second, I violently agree with Toby that automated systems need to build on the standards that people actually use.

Toby chairs the OASIS Web Services Calendar Technical Committee and is an editor of the WS-Calendar specification. If standards whose names begin with WS- give you the heebie-jeebies, take a deep breath. I guess at some point there will be a SOAP profile for WS-Calendar but for now it’s a straightforward set of iCalendar extensions that define intervals, sequences, and relationships.

The web succeeds, in part, because its atomic particle — the hyperlink — can be manipulated both by automated systems and by people. When systems use hyperlinks it’s called RESTful web services. When people do it’s called emailing links. We tend to forget how profoundly the dual nature of the hyperlink binds the web together.

The hyperlink is atomic with respect to location. It says where. The fundamental particle of calendaring — iCalendar’s VEVENT — is atomic with respect to time. It says when. To me it makes perfect sense that smart buildings and energy microgrids will schedule their interactions with us in the same way that we schedule dentist appointments and soccer games.

As I build out the elmcity network, launching calendar hubs in towns and cities around the country, I’ve been gathering examples of excellent web thinking. In Ann Arbor’s public schools are thinking like the web I noted that the schools in that town — and most particularly the Slauson Middle School — are Doing It Right with respect to online calendars. How, I wondered, does that happen? How does a middle school figure out a solution that eludes most universities, theaters, city governments, nightclubs, museums, and other organizations with calendars of interest to the public?

It’s not technology. Slauson Middle School is using the same web services (in this case, Google Calendar) available to everyone.

It’s not budget. The web services required for this solution are free.

It’s a way of thinking. I wrote to Slauson’s principal, Chris Curtis, to congratulate him on the excellent example his school is setting, and to identify the thinker responsible. That thinker turns out to be Chris Curtis himself. And it’s no accident that the implementation pattern on display at Slauson is also evident at Pioneer High. Chris did the same thing there before coming to Slauson.

Now I am not an educator, I only watch from the sidelines. But to me the K-12 “computer skills” curriculum seems uniformed by the kinds of core principles that will make students effective in a web-augmented world. So I asked Chris:

What you’ve done at Pioneer and now Slauson builds on an important conceptual foundation. Do you think that K-12 education could build that foundation?

Here’s what he said:

I agree with the notion that the basic principles of computer science should be generalized more broadly across the curriculum. In many ways, teaching computer and technology skills courses absent a meaningful application of them is ineffective and silly. We wouldn’t teach driver’s education and not let students drive. We don’t teach a “pencil skills class” in which we learn the skills for using this technology tool without an immediate opportunity to apply the skills and then begin to consider and explore the many ways that the pencil and writing change how we organize, perceive, and interact with our world.

This issue gets at the heart of the challenge of technology and education. Often the world seems to divide into separate interest areas: those interested in technology and those interested in education. The result is often to send the technology nerds to a room and make them teach technology and send the other teachers to their rooms and let them teach. In order to be effective at integrating technology into the instructional environment there has to be a merger between a technology interest and and educational interest, within the same person. The awareness of what is possible via technology resources and the desire to perform educational functions can lead to the educator realizing that a task could be done differently, more efficiently, more effectively, with more precision, or in some other manner improved.

Of course the schism that separates technologists from educators also affects practitioners of all kinds. In his most recent essay, Bret Victor meditates on this point:

My piano teacher played the piano. Like, all the time. He had to; it’s not easy to make a living as a musician. Between tours, his band played restaurants, bars, weddings, anywhere they could get a gig. He chose this life because he loved music, and when he taught music, he was teaching what he did. In that way, his teaching was honest.

…

Back in high school, I was taught differential equations by a working engineer. He spent his days at Lawrence Livermore Laboratory, and for whatever reason, chose to spend his evenings at the local community college. Differential equations wasn’t some abstract arcana to him. It was his bread-and-butter, and he apparently found it important enough to share.

…

My information theory professor would teach me information theory in the morning, and then spend the afternoon furthering the field. Sure, what she taught was somewhat elementary by her standards, but she was well aware that this elementary theory was the foundation on which her life’s research was built. It showed, and it stayed with me.

…

Real teaching is not about transferring “the material”, as if knowledge were some sort of mass-produced commodity that ships from Amazon. Real teaching is about conveying a way of thinking. How can a teacher convey a way of thinking when he doesn’t genuinely think that way?

I’m preoccupied with a related question. The way of thinking that I most want to convey is web thinking. Which is, by definition, openly available to anyone who wants to learn. Schools everywhere can observe and emulate what Chris Curtis is doing at Slauson. In so doing they can become practitioners in the way that Chris is. Their students might then see them as practitioners and learn from their examples.

I would be delighted if the elmcity project could help bootstrap that virtuous cycle.

John Ochsendorf, who teaches civil and environmental engineering and architecture at MIT, cares about ancient construction methods and the forgotten knowledge they embodied: rammed-earth walls, braided-fiber suspension bridges, Gothic cathedrals. In an enlightening lunchtime talk a couple of years ago he reviewed what these ancient builders knew and could still teach us if we were willing to learn.

Dave Winer’s Why apps are not the future reminded me of that. The ancient construction method we are forgetting, in this case, is linking. We had already started to forget back in 2004. Then, writing for InfoWorld, I argued that rich Internet apps needed to learn what web apps always knew: RESTful design that exposes internal state as links. Kevin Lynch, who was then chief software architect of Macromedia, accepted the challenge and showed how a Flash app could behave that way.

But while you can bolt RESTful design and linking onto a rich Internet app, or a phone app, these principles aren’t built in. So apps are inherently less open to the sort of lightweight service composition that I’ve championed for a decade, ever since the LibraryLookup project showed me just how easily and powerfully a humble hyperlink can connect two otherwise separate information silos.

The point about service composition and linking is so obvious that we often fail to notice it. Yesterday I booked a hotel in Chicago for an upcoming conference. The organizer sent me an email containing this link for hotel booking:

http://www.marriott.com/hotels/travel/chidm-chicago-marriott/?toDate=1/22/12&groupCode=GROUPCODE&fromDate=1/20/12&app=resvlink

When I followed the link I landed in Marriot’s service with context appropriate to my status as an attendee. Nobody at Marriott specifically designed that capability. It’s just something that’s possible because the site works RESTfully, exposing its state as links that people can hand around.

Now to be honest, you have to be a geek-minded person — like the guy who invited me to this conference — to recognize and exploit this possibility. Most people don’t expect things to work this way, and most web designers don’t try to encourage that expectation.

My fear with rich Internet apps, and now phone apps, is that they tend to obscure an important lesson we never learned well enough in the first place.

It’s been a busy month for rogue plumber Harry Tuttle. When last heard from, he was deploying a filter to fix broken iCalendar feeds served up by the University of Michigan. His next assignment was at Rice University, where the downloadable sports schedules are available only in CSV (comma-separated values) format. Here are the instructions for how to get one of those CSV files into Outloook:

1. Select the File / Import and Export… Menu option.
2. Select “Import from another program or file” and click Next.
3. Select “Comma Separated Values (Windows)” and click Next.
4. Find the desired CSV file and select “Do not import duplicate items” and click Next.
5. Select Calendar and click Next.
6. The next window will have a title that says ‘The following actions will be performed:’above an option to ‘Import “(filename).csv” into folder: Calendar.’
7. Check that box and and click on ‘Import “(filename).csv” into folder: Calendar’From there, click and drag needed fields from the left window into the right window. Its fairly self-explanatory, but the basic corresponding fields are below: SUBJECT to Subject START_DATE to Start Date START_TIME to Start Time END_TIME to End Time DESCRIPTION to Description LOCATION to Location

It’s hard to believe any Outlook user has ever followed those instructions. And if you’re using a calendar other than Outlook, you’re on your own.

So I asked Harry to make a filter that turns those CSV files into iCalendar feeds. It’s specific to that particular format for now, but as we find other examples in the wild we’ll generalize accordingly.

My next challenge involved feed categorization. I’m finding pockets of excellence in various places around the country. As I mentioned last month, the Slauson Middle School in Ann Arbor is one of them. Its calendar looks like a single feed, but is actually a merged set of 11 distinct Google Calendars, including one for Slauson clubs, several for Slauson sports, and one for Slauson’s upcoming Chicago trip.

As curator of Ann Arbor’s community hub I was thrilled to see this kind of self-categorization. It enables me to merge Slauson’s sports and music events with other sports and music events in Ann Arbor. But if it’s rare to find a public calendar in machine-readable iCalendar format, which it unfortunately is, it’s even rarer to find a set of machine-readable public calendars that can be syndicated by category.

I asked Harry to weigh in on this problem too. He’s a do-it-yourself guy who doesn’t like to wait for Central Services to take care of things. “If the sources aren’t providing you with the categories you need,” he said, “maybe we can create them ourselves.” Next thing you know, he’d whipped up a filter that selects subsets of iCalendar feeds by keyword and/or time.

Here’s one example of how I’m using it. Vinology, a restaurant and wine bar in Ann Arbor, publishes upcoming events using a Google Calendar. The feed from that calendar mixes two different kinds of events: restaurant business (Happy Hour) and a jazz schedule (Meri Slaven trio). I’d like to select just the jazz events for Ann Arbor’s music and jazz views.

Harry’s filter solves this problem. Actually, it solves it in two different ways. The first way is to filter Vinology’s feed for events where the SUMMARY or DESCRIPTION fields contain the keyword jazz. The second way is to filter the feed for events that happen after 8PM. Either of these methods produces the jazz-categorized iCalendar feed that Vinology might someday create for itself.

Harry’s filter also solves a related problem. The Berkeley Art Museum / Pacific Film Archive is, as the name suggests, a venue for both art exhibits and film showings. The calendar is, happily, available as an iCalendar feed. Unhappily it mixes art exhibits with film showings and I couldn’t find any times or keywords that would reliably separate the two streams. But then I found a keyword that would. The BAM/PFA website’s URLs look like http://www.bampfa.berkeley.edu/exhibition/faulders_bamscape) and http://www.bampfa.berkeley.edu/filmseries/clouzot). And those same URLs appear in the iCalendar feed. So Harry enhanced the filter to look for those boldedkeywords in the URL field of iCalendar events. Now the BAM/PFA’s film series shows up where it belongs, in the film category of the Berkeley hub.

Senator Ted Stevens got it right. The Internet really is a series of tubes. When they don’t fit together quite the way you need them to, Harry Tuttle can sometimes help.

As we saw last week, the future of community-scale calendaring is already here in some cases but isn’t yet evenly distributed. Consider the Seattle Aquarium’s public calendar. As curator of Seattle’s elmcity hub I’ve found four iCalendar feeds for the Aquarium:

1. An Eventful venue. (iCal feed.)

2. An Upcoming venue. (iCal feed.)

3. A Facebook Page. (iCal feed.)

4. The Aquarium’s own calendar page. (iCal feed.)

If you visit #4, the Aquarium’s calendar page, you’ll see that it does offer an Export as iCalendar link. So why isn’t that link hot here? Because it’s only wired to JavaScript code that imports a snapshot of the data into a calendar program. No URL points directly to the data. That means people can’t subscribe to the feed — and neither can the Seattle hub.

If the Aquarium were truly thinking like the web it would offer its calendar as a first-class web resource addressed by an iCalendar URL. Even better, it would offer one such URL for each view of the calendar page: Educators, Family / Kids, Members Only, Out and About, Speaker Series, Special Events, Summer Camps, Volunteer Orientation.

Since those views aren’t available as feeds I had to fall back to the other three sources. At first I merged all three to create the aquarium view of the Seattle hub. When I compared the results of the merge, though, something didn’t cross-check. An event called Marine Mammal Talk, which showed up in the Upcoming feed, wasn’t in the Eventful or Facebook feeds or on the Aquarium’s own page. Yet Upcoming’s Marine Mammal Talk page says this event was discovered by the Upcoming Robot, has occurred 498 times already, and will occur 77 more times. What’s up with that?

I searched the Aquarium’s site and found the event. Here are the details:

Tue, 15 Nov, 2011 2:43 PM – 2:43 PM

Three weekends of special activities all about the Aquarium’s Marine Mammals – Otters, Harbor Seals & Fur Seals. Join us for special talks, demonstrations and hands-on activities.

Fees: Included with regular Aquarium Admission

Calendar: Programs & Events Calendar

Category: Special Events

Repeats: Weekly on Sunday, Friday, Saturday until 4/24/2011

You can see why the Upcoming Robot was confused. The event’s date is given as Tue 15 Nov 2011. But its recurrence rule says that the last recurrence was back in April of this year.

Of course this fragment of text doesn’t really specify a date or a recurrence rule. The Upcoming Robot just inferred those, erroneously, from unstructured text. And then, like the Sorcerer’s Apprentice, it began pumping those errors onto the web. It has evidently done so for a couple of years and will, if unchecked, continue into 2013.

I don’t blame the Upcoming Robot. Denied access to the structured data it craved, the poor thing scavenged what it could. The lesson here is for the Seattle Aquarium and for all who mean to publish data online. If you don’t establish yourself as the authoritative source for that data then others will step in to do it for you. And they are liable to get it wrong.

As I review and improve the elmcity hubs in selected cities, I am again reminded of William Gibson’s wonderful aphorism: “The future is already here, it’s just not evenly distributed.” Yesterday we saw that the future of community calendars hasn’t yet arrived at the University of Michigan. But today I was delighted to see that it has arrived, in a big way, for the Ann Arbor public schools. Almost all of them, it turns out, are making good use of Google Calendar to publish machine-readable calendar information. This morning I rounded up thirty of those calendars and added them to Ann Arbor’s elmcity hub, bringing the total number of feeds from 194 to 224.

Here’s the breakdown of the 309 events from the grade schools:

And the 322 events from the middle schools:

And the 966 events from the high schools:

Among grade schools, Slauson is notable not only for the number of events but for the exemplary self-categorization applied to them. When you click the Google Calendar subscribe button on the Slauson calendar page here’s what you’ll see:

This is a best practice I wish everyone would adopt. It illustrates the seventh of my seven ways to think like the web: #7: Reuse components and services. The Slauson calendar is both a user of other services (the district-wide calendars) and a provider of services. And as a provider, it understands the idea of componentization. In an era of abundance it costs no more to create and manage a dozen calendars, using free services like Google Calendar and Hotmail Calendar, than to jam everything into a single calendar. The benefits are manifold. They include:

Delegation

In most schools and businesses, maintenance of “the” public calendar is one person’s job. That person becomes a bottleneck. When you recognize that logically there isn’t one public calendar, but instead several or many, each with its own appropriate maintainer, then you can break that bottleneck.

Precision

A parent who subscribes to a single undifferentiated school calendar may be overwhelmed by the flow. Parents of kids who are in music programs, or on sports teams, or who are going on the Chicago trip, should be able to focus on those activities.

Scope

Slauson Middle School is part of the larger Ann Arbor community. When Slauson’s calendars are self-categorized, they can align with community-wide views. Here’s a picture of some sports-related activites in Ann Arbor on November 17th:

Slauson’s publication of a set of self-categorized machine-readable calendar feeds enables it to represent its sports activities on a city-wide timeline that includes, in this particular view, events from the Ann Arbor Triathlon Club, the Wolverines basketball teams, and Ann Arbor’s kickball Meetup group.

Well done Slauson! And kudos to all the Ann Arbor public schools. You have become web thinkers. I hope schools everywhere will learn from your example.

Here’s one of my favorite scenes from the movie Brazil,

Sam Lowry (Jonathan Pryce): Are you from central services?

Harry Tuttle (Robert De Niro): Hah! They’re a little overworked these days. Luckily I intercepted your call.

Sam Lowry: Can you fix it?

Harry Tuttle: No, but I can bypass it with one of these.

Back in 2003, my InfoWorld column APIs, protocols, and rogue plumbers made three points:

1. The web of data shouldn’t require the services of Harry Tuttle.

2. Unfortunately it still does.

3. Fortunately the web’s architecture (a series of tubes) enables Harry to intervene when he must.

Those points are all still valid in 2011. From time to time I still have Harry Tuttle moments. Today’s involved the campus events system at my alma mater, the University of Michigan. In the current phase of the elmcity project I’m rebuilding hubs in various cities in order to dramatically beef up the number of feeds and quality of tagging. Ann Arbor’s hub was conspicuously lacking feeds from the University of Michigan. When I investigated I found that the central service, http://events.umich.edu, does offer iCalendar feeds. Yay! You can’t take that for granted, many if not most schools don’t make their public calendars available in a machine-readable way. Unfortunately there’s a problem with the feeds produced by the UM service. They’re invalid. You can’t load them into an iCalendar-aware program like Google Calendar or Outlook, and the elmcity engine can’t aggregate them.

I reported the problem to central services and have been awaiting a fix for some time. Today, because I wanted to get my hands on that data, I unleashed Harry Tuttle. There are two major problems with the iCalendar export from events.umich.edu. First, the lines of text aren’t properly folded. Second, the timezone properties don’t refer to a timezone definition. So I made a filter that fixes these major problems (plus some other minor ones). Here’s what the filter does:

Original: http://jonudell.net/data/failed-ics-umich.ics.txt. Validation result: Unparseable.

Line-folding fixed: http://jonudell.net/data/fixed-ics-umich.ics.txt. Validation result: Better.

VTIMEZONE added: http://jonudell.net/data/clean-ics-umich.ics.txt. Validation result: Valid.

Then I used the filter to add a bunch of feeds to the hub. Here’s one for the Taubman Health Services Libraries, and another for the Gerald R. Ford Presidential Library. These merge with other library feeds, notably from the Ann Arbor District Library, in the hub’s top-level library view.

Now, where do I fill out that twenty-seven-B-stroke-six form?

From Wikipedia’s Brazil_(film) page:

The reference to form 27B-6, without which no work can be done by repairmen of the Department of Public Works, is a reference to George Orwell, who lived at 27B Canonbury Square Apartment 6, while writing Nineteen Eighty-Four.

In a pair of recent entries, Semantic web 101: Say what you mean and The long tail of the iCalendar ecosystem, I’ve begun to report on what I’m learning about the state of the iCalendar ecosystem as I work in parallel on the elmcity project and on the iCalendar Validator. Today I’ll focus on just one of a number of issues I’ve run into. Consider these two screenshots:

google calendar	hotmail calendar

On the left you see Google Calendar displaying two calendars, each representing a single event — the Brower Youth Awards on October 18 at the Herbst Theater in San Francisco — in a different way. On the right you see Hotmail Calendar displaying the same two calendars. The event will happen at 5:30 Pacific time on the 18th. I found it on the Berkeleyside site whose events page offers a companion iCalendar feed.

If you load that feed into both Google Calendar and Hotmail Calendar, and if your calendars are set to Eastern time, you’ll see what’s shown above. If your calendars are set to another timezone the times will be shifted but the pink ones still won’t match.

The green ones should always match and should always be what you’d expect. For me, looking at a 5:30 Pacific event through the lense of calendars set to Eastern, I’d expect both calendars to display 8:30.

What’s the difference between the pink calendar and the green calendar? Here’s the pink one. It’s just the original calendar reduced to a single event. Like the original it declares its timezone using the nonstandard X-WR-TIMEZONE property:

BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
CALSCALE:GREGORIAN
PRODID:-//Refresh Web Development//Helios Calendar//EN
X-FROM-URL:http://www.berkeleyside.com/BerkeleysideCalendar/events/
X-WR-RELCALID:Berkeleyside
X-WR-CALNAME:Berkeleyside
X-WR-TIMEZONE:America/Los_Angeles
BEGIN:VEVENT
URL;VALUE=URI:http://www.berkeleyside.com/BerkeleysideCalendar/events/index.php?com=detail&eID=6
DTSTART:20111018T173000
DTEND:20111018T210000
SUMMARY:Brower Youth Awards 2011
LOCATION:Herbst Theater - 401 Van Ness \, San Francisco\, CA US 94102
END:VEVENT
END:VCALENDAR

And here’s the green one. Again it’s a derivation of the original calendar that reduces to a single event. But it also declares its timezone in the standard way, using a VTIMEZONE component and then referring to it using the TZID parameter of the DTSTART and DTEND properties:

BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
CALSCALE:GREGORIAN
PRODID:-//Refresh Web Development//Helios Calendar//EN
X-FROM-URL:http://www.berkeleyside.com/BerkeleysideCalendar/events/
X-WR-RELCALID:Berkeleyside
X-WR-CALNAME:Berkeleyside
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
URL;VALUE=URI:http://www.berkeleyside.com/BerkeleysideCalendar/events/index.php?com=detail&eID=6
DTSTART;TZID=America/Los_Angeles:20111018T173000
DTEND;TZID=America/Los_Angeles:20111018T210000
SUMMARY:Brower Youth Awards 2011
LOCATION:Herbst Theater - 401 Van Ness \, San Francisco\, CA US 94102
END:VEVENT
END:VCALENDAR

As we see in the picture above, the event time on the green calendar shows up the same way in both Google Calendar and Hotmail Calendar. It should also be the same in any calendar program that supports the iCalendar standard.

The event time on the pink calendar, though, is a up for grabs. Calendar programs that strictly follow the iCalendar standard should ignore X-WR-TIMEZONE and always display the local time, 5:30PM, which will be right for people in the Pacific timezone and wrong for everybody else. Hotmail Calendar does this. Programs that use X-WR-TIMEZONE, on the other hand, can render this calendar just as they would render a standard calendar. Google Calendar does this.

Why do I care? I have to decide whether the elmcity service will or won’t consider X-WR-TIMEZONE to be meaningful. The service is based on DDay.iCal, the same standards-based parser that powers the iCalendar Validator. So when the the service reads the pink calendar, and renders it for users in Berkeley, it will do the wrong thing from their point of view.

To do the right thing for Berkeley it would need to do the wrong thing by iCalendar: transform the nonstandard X-WR-TIMEZONE property into a standard VTIMEZONE component, and then transform all the dates so that they refer to the VTIMEZONE’s TZID. In order to create that VTIMEZONE component, it would interpret X-WR-TIMEZONE value as a TZID (timezone ID) from the Olson database. A Unix-based service would look up the TZID in Olson, find the rule for the timezone — i.e., offsets from GMT for standard time and daylight savings time, and when to appy them — and express the rule using VTIMEZONE syntax. A service running on Windows Azure, like mine, would instead need to map the Olson name to a Windows timezone name, look up the rule using a Windows API, and then express the rule in VTIMEZONE syntax.

Of course this is a slippery slope because, in the end, I’m only guessing what X-WR-TIMEZONE is supposed to mean. Here’s Rick DeNatale engaging in the same kind of guesswork:

Someone pointed me to this icalendar file of Australian holidays for a test case:

http://icalx.com/public/rohanl/Australian32Holidays.ics

This contains NO VTIMEZONE components, but does have the calendar property: X-WR-TIMEZONE:Australia/Sydney

Googling indicates that this is a non-standardized property, but it seems to be used by several calendar apps including Apple’s ical.app
and Google calendar.

I know that it’s non-standard, but it seems to be somewhat important for interoperability. I’m looking for some kind of information about
what it means in general.

It seems to indicate a default tzid for the whole calendar. In the absence of timezone components I’m not sure how to interpret the tzid, though.

Australia/Sydney IS a time zone identifier in the Olsen database, is it standard practice to use olsen tzids in X-WR-TIMEZONE calendar attributes?

Fortunately I can bring some data to bear on this question. Thanks to the iCalendar Validator I can analyze public calendars produced by a variety of iCalendar producers. In The long tail of the iCalendar ecosystem I listed the names of about 600 producers seen recently by the Validator. Of those, about 100 use X-WR-TIMEZONE instead of VTIMEZONE, and 70 of those 100 use local rather that UTC date syntax which implies they are depending on X-WR-TIMEZONE for correct interpretation of those dates.

Note that Google Calendar, the 800-pound gorilla in this space, is not one of those 70 producers. When it writes iCalendar format it uses both X-WR-TIMEZONE and VTIMEZONE; the latter ensures that Google Calendars can be understood properly by standard parsers that don’t support X-WR-TIMEZONE. The 100 producers I’m talking about, though, are using only X-WR-CALENDAR in a way that suggests they expect a nonstandard transformation. The fact that Google Calendar performs that transformation is, of course, a major reason why producers would expect it to happen everywhere.

Should X-WR-TIMEZONE be standard? That’s debatable. It would certainly make life easier for iCalendar producers. They could just mention a timezone rather than having to extract its rule from their operating systems and express the rule in VTIMEZONE syntax. One of the reasons for the success of RSS and Atom, after all, is that it’s always been easy to whip up an RSS or Atom feed which you can then check with the Feed Validator. An analogous simplicity for iCalendar producers would help grow the iCalendar ecosystem.

On the other hand if an iCalendar feed were to only mention a timezone without fully defining it, then the consumer would have to do the work that the producer didn’t. That’s problematic as Doug Day, author of the iCalendar Validator, notes in a recent email exchange:

Unfortunately, without including the actual time zone information in the calendar (i.e. via VTIMEZONE), you can’t be sure that the date/times you’re representing are accurate, even when using X-WR-TIMEZONE. For example, if you’re on a Windows XP machine that hasn’t been updated in 5 or 6 years, your system time zone information will be inaccurate. However, if the VTIMEZONE were included in the calendar, it would remain accurate, even on an older machine with out-of-date time zone definitions. Also, in order to interpret X-WR-TIMEZONE, you’d need to be in an environment where interpreting Olson time zone is realistic (easy on Linux, harder on Windows). I know a global, online time zone registry is in the works, but I don’t think it’s to a point where it’s useful, yet.

You might wonder why all this timezone stuff is even necessary. After all, an iCalendar feed can simply omit VTIMEZONE (and/or X-WR-TIMEZONE), express dates and times in UTC (Coordinated Universal Time), and use UTC syntax for all dates and times. Why not just do that? I asked Doug Day about this a while ago, and here was his reply:

The biggest problem is with recurring events and daylight/standard time transitions. For example, consider the following (all hypothetical):

1. I live in Salt Lake City, Utah

2. I want to schedule a meeting, starting on September 7, 2009 at 9:00 A.M, which recurs every month on the first Monday.

3. Some of the people attending this meeting live outside my current time zone.

So, here are the occurrences you’re ultimately after:

- September 7, 2009 – 9:00 A.M. (3:00 PM UTC)

- October 5, 2009 – 9:00 A.M. (3:00 PM UTC)

- November 2, 2009 – 9:00 A.M. (4:00 PM UTC)

As you can see, once the time changes from daylight back to standard time, so does the UTC representation of that time. So, if you had scheduled your event in UTC time, when the time zone changes, your event time will actually have changed (to 10:00 A.M., rather than 9:00 A.M.)

For this reason, among others, it’s always best to include time zone information whenever available. Traditionally, it’s been pretty difficult to include that information, and it’s more often left out than included.

So what shall I do with X-WR-TIMEZONE? I’ve decided to support it experimentally. If you start a new hub on the elmcity service, the default is to ignore X-WR-TIMEZONE. But if your hub has important sources that depend on it, as Berkeley does, then you can override the default so the times will be as you expect.

Meanwhile we’re going to update the iCalendar Validator to warn producers about this issue. There’s nothing technically invalid about a calendar that uses X-WR-TIMEZONE without VTIMEZONE. To a parser that strictly interprets the RFC5545 standard, that property is just a name that’s “reserved for experimental use.” But as has always been true of the RSS/ATOM Validator, the iCalendar Validator aims to deliver useful real-world guidance. Producers that use X-WR-TIMEZONE alone to declare a timezone should know that while it may often yield expected results, it’s not guaranteed to do so. It would be better to use a standard VTIMEZONE.

A couple of months ago I began saving the iCalendar files that are submitted to the iCalendar Validator. Today I extracted a list of unique names of iCalendar producers along with associated counts of the number of calendars validated for each. Here they are, with Google Calendar at the head and a classic long tail distribution of almost 600 other iCalendar producers. (You can also see them elsewhere by name as well as by count.)

The next step will be to analyze how well these these producers conform to the validator’s interpretation of the iCalendar spec. But the list itself forms an interesting data set. We know intuitively that, after 12 years of evolution, the iCalendar ecosystem must have become broad and diverse. Here’s a nice illustration of that breadth and diversity.

(Note that these counts don’t necessarily reflect the real distribution of iCalendar producers. The iCalendar Validator is closely associated with the elmcity project, so certain producers used heavily there — DDay.iCal, EVDB, Meetup — are overrepresented. On the whole, though, I’d guess this is a reasonable proxy for the distribution of producers.)

If you think that the semantic web is just some kind of geek rapture, like the singularity, I can understand why. As with Zeno’s Paradox we’re always advancing but never arriving. Unlike the singularity, though, I do expect a semantic web to emerge in my lifetime. The latest initiative is schema.org, sponsored by Google, by Yahoo!, and by my employer, Microsoft. Schema.org recapitulates prior efforts to define how webmasters can mark up web pages to include structured data. I hope that this time, thanks to collaboration among the major search engines, we’ll finally cross the activation threshold.

Meanwhile I’ve been toiling in another part of the semantic web. I’ve been trying to get webmasters people to understand why and how to publish calendars as machine-readable structured data in addition to human-readable text. If you’ve followed the elmcity saga you know I’m on a crusade to make more and better use of the Internet’s venerable standard for exchanging structured calendar events: iCalendar.

It’s been a struggle. Almost every website run by a school, club, business, or town has an Events page. Those pages are, almost always, data siloes: HTML or PDF files that can be read by people but cannot be processed by machines. Only rarely do such pages offer links to iCalendar feeds served up by Google Calendar or Drupal or Hotmail Calendar or some other service capable of producing such feeds. So when elmcity curators discover one of these rare feeds, it’s cause for rejoicing.

Sadly that joy is sometimes short-lived. A surprising number of iCalendar feeds just plain don’t work. That’s why I invited Doug Day to create the iCalendar Validator, a service that helps producers of iCalendar feeds conform to the specification. It’s always painful when I have to explain to a curator that the shiny new feed they’ve discovered doesn’t conform and won’t deliver events to the hub.

Here are three sources of iCalendar feeds that, I’ve recently discovered, don’t work.

1. The University of Michigan’s UM Events. It’s a major hub that serves many campus websites. As far as I can tell, all of those sites are providing feeds with malformed descriptions. Here’s an example of the problem and the fix. (iCalendar producer ID: UM//UM*Events)

2. CiviCRM is a “free, libre and open source software constituent relationship management solution.” Its feeds also have malformed descriptions. Here’s an example of the problem and the fix. (iCalendar producer ID: CiviCRM//NONSGML CiviEvent iCal)

3. Drupal is a popular open source content management system. I’ve seen Drupal feeds used successfully, but today I found one that fails for two reasons: a malformed recurrence rule, and a missing timezone definition. Here’s an example of the problem and the fix. (iCalendar producer ID: Drupal iCal API)

These problems are minor and would be easy to resolve. I’ll try to contact the authors of these iCalendar producers; if you can help put me in touch I’d appreciate that. I’m also going to look through the logs written by the iCalendar Validator, compile a list of producers of invalid feeds, and try to contact them as well.

Where’s the connection to the semantic web? At the end of the day, as RSS/Atom validator co-creator Sam Ruby likes to say, “It’s just data.” But structured data, whether it conforms to the dozen-year-old iCalendar standard or some newfangled microdata standard, is easily screwed up. And the consequences of screwups are often silent. Services that were looking for that structured data find nothing, mutter to themselves, and move along.

As we collectively create the semantic web we’ll need to make sure that the structured data we intend to publish really says what we mean.

If you follow this sort of thing, you already know about IFTTT. It’s a new web service that enables non-programmers to compose other web services. The acronym expands to If This Then That, and here are some ways you can use the metaphor:

If I am tagged in a Facebook photo, then save the photo to Evernote

If the library sends email saying my book on hold is ready, send me a text message

My colleague Scott Hanselman says this is bloody brilliant and I agree. It’s the next step in a journey that began for me back in 1999 when I mashed up Alta Vista’s search engine with Yahoo! directories to measure the mindshare of sites by category. More recently Yahoo! Pipes made service mashing easier for non-programmers. Now IFTTT enables everyone to play. Wonderful!

So why am I less enthusiastic about IFTT than I thought I would be? There are two related reasons. First, here’s what IFTTT says when you ask it to activate its Twitter channel on your behalf:

This application will be able to:

• Read Tweets from your timeline.
• See who you follow, and follow new people.
• Update your profile.
• Post Tweets for you.
• Access your direct messages.

Excellent! This is an example of OAuth, a protocol that enables you to delegate powers to IFTTT without giving up credentials. Most of the services you can use with IFTTT support OAuth, and that represents another huge step forward for the web.

What if I only want to give IFTTT the power to tweet on my behalf, though, and not give up access to my private direct messages? More generally, how can I think about the tradeoffs involved in delegating all versus some versus no powers to IFTTT, across a range of services I might authorize it to use on my behalf?

This leads to the second and broader concern. If I’m not paying for the product, I am the product. As is true for many free services on the web today, I have no contractual relationship with IFTTT. I pay for the service it provides by surrendering access to my data. OAuth helps me negotiate how much access, but if I give up none then IFTTT is powerless.

Why, though, can’t I pay for the product instead of being the product? I want IFTTT to work for me, I want to pay for the service, and in return I want it to promise never to keep or use any of the data it exchanges on my behalf. I realize this may not be a popular option anytime soon. But it’s time to start the conversation. Services don’t only want to be free, they also want to be valuable. That’s rarely a choice nowadays. It needs to become one.

Yesterday’s stream of notifications brought two links paired with invitations for me to comment. The first link points to a NY Times story about how AVOS, the new owner of Delicious, plans to remake that service. Chad Hurley:

The home page would feature browseable “stacks,” or collections of related images, videos and links shared around topical events. The site would also make personalized recommendations for users, based on their sharing habits. “We want to simplify things visually, mainstream the product and make it easier for people to understand what they’re doing,” Mr. Hurley said.

The second link points to a blog post from Mark Surman, executive director of the Mozilla Foundation, who wants to teach the world to code:

This has been the premise behind much of what we have done with Mozilla Drumbeat: people who make stuff on the internet are better creators and better online citizens if they know at least a little bit about the web’s basic building blocks. Even if they only learn a little HTML, the web gets better.

I wish I could broker a conversation between Chad Hurley and Mark Surman. If mainstream folk used Delicious and understood what they’re doing when using it, they’d understand themselves to be makers of things on the Internet. The things they make are custom information systems. They make them by writing code, but not in the languages of HTML or JavaScript. Instead they use tag vocabularies that produce and consume web services. And services, I argue, are the most fundamental of the web’s basic building blocks.

Here is an example: http://www.delicious.com/judell/del.icio.us. Ostensibly it’s a list of dozens of articles I’ve written over the years about what I mean when I say that Delicious enables non-programmers to code and use web services. But it’s not just a list. I think of it as a web service. One aspect of the service provides the list in HTML format for people to read in browsers. Another provides the list in RSS format that enables cooperating services to watch the list and react when new items are added. Another enables the list to combine with other lists. Here, for example, is a subset of my Delicious-related articles that are also related to the elmcity project: http://www.delicious.com/judell/del.icio.us+elmcity.

More than anything before or since, Delicious empowers me to manage web resources — both personally and socially. Once those resources were mainly things we found on the web. Now they’re also things we make on the web. I hope the forthcoming Delicious makeover will help people understand it to be a tool for creating, mixing, and sharing web resources. And I hope it remains the sort of open web tool that Mozilla Drumbeat wants to popularize.

Last September we bought a new dishwasher to replace the old one that had failed. It was a reluctant purchase. We’d actually gone a couple of years doing dishes by hand, partly because we’ve been so disappointed by the modern generation of appliances. When the salesman at Sears mentioned their free 5-year preventive maintenance program, though, we decided to opt in. The plan entitles you to an annual appointment with a dishwasher tech who will come by to inspect your machine and do what’s needed to keep it in good working order.

Stoves, dishwashers, and vacuum cleaners shouldn’t require this kind of life support. I’m not going to win that argument, but I can at least make effective use of the life support service. Doing so requires a bit of a hack, though. Why? Remembering something annually, on a certain date, for five years running, isn’t the sort of thing that humans do well. It’s a task that begs for automation.

Of course Sears could remind me, in September of 2011, 2012, 2013, 2014, and 2015, about my dishwasher checkup. It would be easy for them to do. They’re a mega-corporation with a mighty IT department that can easily perform this feat of magic. But Sears doesn’t want to remind me. They’re betting that I’ll forget, as I’m sure most people do.

So they sounded quite surprised when I called yesterday, on the anniversary of my dishwasher purchase, to schedule my appointment. How did I do it? By casting this magic spell:

BEGIN:VEVENT
DTSTART;VALUE=DATE:20110902
DTEND;VALUE=DATE:20110903
RRULE:FREQ=YEARLY;COUNT=5
SUMMARY:sears dishwasher
END:VEVENT

That’s an event recorded by my calendar program. It happened first on September 2, 2011, and will happen again yearly for five years. I cast my spell using Outlook but you can do the same thing using any calendar program: Google Calendar, Apple iCal, Hotmail Calendar, Lotus Notes, many others.

When I talk about iCalendar feeds in the context of the elmcity project, I tend to focus on the idea that these are data feeds, which they are. But iCalendar has a special property. Some of that data is actually code. In this example, here is the one line of code:

RRULE:FREQ=YEARLY;COUNT=5

In iCalendar lingo, RRULE means recurrence rule. From Outlook’s or Google Calendar’s point of view the rule is a tiny program and they are the operating systems that run it. Here are some other RRULES that calendar applications enable you to express:

every Tuesday at 10AM

every first Sunday until October 2011

every September 2 for five years

You don’t have to do it this way. You can, instead, record each event separately. Even though calendar software makes it quite easy to define repeating events, I’ve got a hunch that in the dishwasher scenario a lot of people would use data (five discrete events) rather than code (one event that expands to five). Why? For the same reason that many people will manually adjust the font size used for every paragraph in a document, rather than creating a rule that governs all the paragraphs.

My list of core principles for web thinkers doesn’t yet include the idea of code that generates data. I’ve found it plenty challenging just to get people to think about the nature of their data, about how they do or don’t control it, and about how it does or doesn’t flow through networks.

But if we want everyone to be able to automate tasks that otherwise get done poorly or not at all, we probably do need to find a way to teach patterns like code-generated data and Don’t Repeat Yourself more broadly than just to students of computer and information sciences.

If you want to read a lovely short essay on calendar software and our relationship to the future, I highly recommend Paul Ford’s Tickler File Forever.

As the environments in which we live and work become augmented by networked information systems, it behooves us to learn something about how those systems work — and about how to work those systems. “But I don’t need to know how my car works,” you say, “why do I need to know how my computer works?” You don’t. That’s the wrong analogy.

Our civilization’s enabling infrastructure exists only because we’ve created many different kinds of specialized knowledge, and because we — the knowers and practitioners of these specialties — participate in networks of exchange. It’s a good thing that most people don’t need to know how to repair faulty disc brakes or laptop hard drives.

It would be really bad, though, if most people didn’t know anything about the physical laws that keep the car’s rubber on the road, or the social laws that govern traffic intersections. That’s the level at which people need to know about networked information systems.

A lot of what the creators of those systems know isn’t at that level. A few things are, though, and I’ve been working up a list of things creators know that users could and arguably should know. From time to time I think of adding another item to the list. So far, though, everything that comes to mind is already, in some essential form, on the list.

The latest example is a rule known as Don’t Repeat Yourself. Programmers invoke DRY when they squeeze redundancy out of code or data. Where does the duplication come from? Information systems, like human cultures, evolve by copying parts of themselves — and of one another — and then by modifying the copies. That’s the natural and healthy trend toward diversity. But systems and cultures also evolve by converging on a core of shared commonality. That’s a natural and healthy countervailing trend. The most successful copies need to merge, at the right level of generality, with the core.

These two styles are in dynamic tension. On the C2 wiki, where Ward Cunningham has for many years hosted a conversation about the core principles that govern information systems, the topics DontRepeatYourself and PrematureGeneralization express that tension. It’s good to DRY things out, but bad to generalize prematurely.

Does that rule belong on the list? Maybe. I wish programmers weren’t the only ones feeling and responding to the tension between DRY and WET¹. Anyone who aims to automate work, for example, needs to learn how to walk the WET/DRY tightrope.

But maybe wringing out redundancy isn’t the essence of DRY. Here’s what the noted software pragmatist Andrew Hunt says on the DontRepeatYourself wiki page:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

That’s an assertion about the value of authority, not the problem of duplication. And the first rule on my list already says: “Be the authoritative source for your own stuff.” In the context of the elmcity project, which is the incubator for the list, that means if you’re promoting a calendar event online, you ought to publish a single authoritative version of that calendar entry to a system that’s accountable to you. You shouldn’t have to repeat yourself by making copies of the calendar entry in systems that aren’t accountable to you.

When you’re the authoritative source for some set of facts, and when you inject those facts into the networked information system called the web, Don’t Repeat Yourself is a great principle to keep in mind. I’ll be delighted if you do. But it’s the underlying core principle of authority that I most want you to embrace. When you’re the authoritative source for your own stuff, you’ll naturally tend to stay DRY.

¹ There’s no agreed-up expansion of DRY’s opposite, WET. Here’s mine: Watch, Emulate, Transcend.

I’ve been mulling Michael Schrage’s recent essay, Why You Should Automate Parts of Your Job to Save It, since I read it last week. Here’s the conclusion:

What is the most important thing you do on your job? What portion of that could be turned into an app that anyone in your organization could effectively use? What portion of that could be automated and fed directly into the larger system with only minimal review by you? What’s the least valuable but essential part of your job? Why aren’t you figuring out ways to automate it on your iPad or Android?

People with the best answers will likely discover they also have the best job security.

I agree with the premise. But something kept bugging me about the argument and today I realized what: the gadget focus. We’ve seen this before. Remember when computers in the schools were the answer? Now it’s smartphones and tablets in the workplace. But these are all just access devices. We focus on them because they seem more real than the networks they connect us to. It’s easy to see that devices made of metal and plastic are tools. It’s much harder to see that networks made of data formats and application protocols and communication topologies are tools. But information networks matter more than the devices we use to access them, or the applications that run on those devices. The key to the automation of knowledge work that Schrage righly prescribes isn’t learning how to use smartphones or tablets. Rather, it’s learning and then applying core principles that govern information networks.

Sadly we don’t teach these principles. Not even, in any systematic way, to information technologists. And certainly not to the bank loan officers and nurses and “iPad-wielding waitresses” in Schrage’s essay. Can it be done? I don’t know but I think I’d enjoy trying.

A couple of years ago, when I needed to call someone, my phone wasn’t where I thought I’d left it. This was a problem. Back then I often called this person but I had no idea what his phone number was. Why would I? Remembering numbers is a job I delegate to my phone.

And finding my phone when I forget where I left it is a job I delegate to other phones. But when I reached for my cordless office phone it wasn’t where I thought I’d left it either. No problem. Finding my cordless when I forget where I left it is a job I delegate to its base station which is tethered and can’t wander.

So I walked over to the base station, paged the cordless, found it under a pile of papers, used it to call my cellphone, found it under another pile of papers, and used it to call my friend. Then I paused to reflect on technological augmentation.

Now don’t get me wrong, augmentation is a wonderful thing. Even fish, we have recently learned, use tools. I love delegating mental chores to devices and to the cloud. I am not a Luddite. But I am starting to think more about when not to delegate, and why not.

My mom is almost 90. She’s always known the numbers of the friends she often calls, and she still does. That’s a good thing because macular degeneration makes it impossible to read the numbers in her phone’s directory and difficult even to read the numbers she’s printed extra-large in her address book.

I’m not planning to memorize phone numbers. But I am exploring ways to usefully exercise the memory muscle. Just because I can look everything up doesn’t mean that I always must. Some kinds of things are worth keeping in the cache. But which?

In Behind the Dream Clarence Jones, Martin Luther King’s lawyer and adviser, writes:

What amazed me was that there was absolutely no reference material for Martin to draw upon. There he was [in the Birmingham jail] pulling quote after quote from thin air. The Bible, yes, as might be expected from a Baptist minister, but also British prime minister William Gladstone, Mahatma Gandhi, William Shakespeare, and St. Augustine.

If there’d been a web in 1963, and if MLK could access it from that jail, would the Letter from a Birmingham Jail have turned out differently? Would the differences have mattered? I’m not sure. It’s interesting to note that the quotes Clarence Jones seems to recall being in the letter aren’t all there. I don’t find Gladstone, Gandhi, or Shakespeare. I do find, along with St. Augustine, Socrates, Thomas Aquinas, Paul Tillich, Abraham Lincoln, Thomas Jefferson, T.S. Eliot and others. Does the discrepancy matter? Not really. What Jones remembers, and what I will now remember, is that MLK could remember, not precisely what he did remember.

Clarence Jones continues:

I have often said the sheer processing power of Martin’s mind left me awestruck. His dexterity with memory and words ran along the lines of the cut-and-paste function in today’s computer programs. The “Letter from a Birmingham Jail” showed his recall for the written material of others; his gruelling schedule of speeches illuminated his ability to do the same for his own words. Martin could remember exact phrases from several of his unrelated speeches and discover a new way of linking them together as if they were all parts of a singular ever-evolving speech. And he could do it on the fly.

Jones tells us that it was he who wrote the draft of I Have a Dream speech. And that Martin Luther King began to deliver it more or less as written. Then Mahalia Jackson called out, “Tell ‘em about the Dream, Martin.” So he did. He set aside the text that Jones had written, and that he had edited the night before, and gave an impromptu speech for the ages. Among many factors that conspired to make that possible were the processing and memory powers that Jones describes.

As our relationship to devices and the cloud reorganizes our brains, those powers are changing. But we’re not just passively experiencing these changes. We are actively co-creating the powers of our individual brains and of the collective brain we’re becoming.

When I was a kid I used to memorize poetry. A couple of years ago I was feeling no need to do anything like that ever again. But now I find myself wanting to exercise the memory muscle. I’ve decided that certain kinds of things belong in the unaugmented part of my brain. Among them, for me, right now, are fingerstyle guitar arrangements. My repertoire has grown slowly over the years, and it’s been a challenge to learn new arrangements. Lately I’ve been working to improve my ability to memorize them, though, and the effort feels intrinsically rewarding.

I’ve also noticed that one of the best talks I’ve ever given was the product of careful memorization. I think I see now why Ignite talks and Moth stories are done live without notes. I love having an outboard brain. But until we get the implants, we’ll want to keep the onboard one humming.

When I left the pageview business I walked away from an engine that had, for many years, manufactured an audience for my writing. Four years on I’m still adjusting to the change. I always used to cringe when publishers talked about using content to drive traffic. Of course when the traffic was being herded my way I loved the attention. And when it wasn’t I felt — still feel — its absence.

There are plenty of things I don’t miss, though. Among them is the obligation to be an aggressive early adopter of (and opinionator on) every new tech fad. Now I can hang back, wait for the fads to spread beyond the geek echo chamber, and watch how my civilian friends, family and acquaintances react to them. Since none of the civilians I know have moved to Google+, I can’t gauge their reactions yet. While waiting for some of them to jump into the pool I’ve dipped a toe in the water, considered my own reaction to the New Thing, and compared it to the collective reactions of the geek tribe.

Mine seems atypical: I’ve reached into a corner of my closet, pulled out the RSS reader I left there, and used it to find nourishment that online social networking seems no longer to provide. Last night’s 17-course meal was a selection of recent essays by Gardner Campbell, Brian Dear, Lorianne DiSabato, John Faughnan, Paul Ford, Cliff Gerrish, Ned Gulley, Eugene Eric Kim, Adina Levin, Hugh McGuire, Cameron Neylon, John Quimby, Antonio Rodriguez, Scott Rosenberg, Doc Searls, Ed Vielmetti, and Ethan Zuckerman.

These writers are among many who write because they want to and because they can. They write in their own online spaces which I follow in my RSS reader. When I seek nourishment from them I can go directly to their spaces. No business models drive me there. Often, to be sure, I have been led there by way of a comment on one or another of the social networks. That had become so common that I came to accept a lot of distracting chatter as the price of discovering things to read. But Google+ seems to be the camel’s-back-breaking straw. The price has gone too high. So I’m rediscovering what made the blog network so thrilling to me a decade ago: unmediated access to people writing for the love of it in their own online spaces. Distracting chatter has its uses. But it’s optional.

Guy Sorman’s The End of Green Ideology begins:

The meltdown at the Fukushima nuclear plant has sent political aftershocks racing around the globe. More often than not, however, the shocks have been ideological, with no basis in science.

I’ve been reexamining the case for nuclear power for a while now. I still think we’ll probably need to keep developing it, improving it, and including it in the mix. But I’m willing to be convinced otherwise and I’ll watch Germany’s forthcoming nuclear exit with great interest. In any case, I agree with Guy Sorman: let’s make sure our energy future is based on science not ideology.

So I was puzzled to find this assertion in the middle of his critique of green ideology:

If California were to rely on solar power for its electricity consumption, the entire state would have to be covered with photovoltaic cells.

Really? That seems wrong. According to an infographic called Surface Area Required to Power the World (with zero carbon emissions and with solar alone), assumptions and methodology here, it looks like a California of solar panels could more than power the world.

For reasons Sorman alludes to — cost, environmental impact — it probably wouldn’t make sense to build a California of solar capacity, regardless of how it’s distributed around the world. But let’s just focus on the assertion that California would need all of itself to keep the lights on using solar photovoltaics. Is that really wrong? If so, how wrong? What’s the right answer? And how can we find out?

Before the advent of WolframAlpha I would sometimes idly speculate about these kinds of questions, but I wasn’t able to effectively analyze them. Now I can. From various sources, I think I know that a PV panel can produce on the order of 10 watts per square foot. So we can ask WA directly:

area of california in square feet * 10 watts per square foot = 45.64 TW

How can we think about 45 terawatts? I recall from Saul Griffith’s Climate Change Recalculated that the whole world runs on about 16TW. This confirms what I gleaned by eyeballing the infographic: Solar California could more than power the world.

But where does Saul’s 16 TW number come from? Can we cross-check it? The infographic assumes that we were running at 500 quadrillion BTU in 2008, projected to rise to around 680 quadrillion BTU in 2030. The US Energy Administration’s International Energy Outlook 2010 cites similar numbers. I’m not good at unit conversions, but WA is:

500 quadrillion btu in watt years / 1 year = 16.73 TW

This matches so closely that I wonder about the provenance of the number. Do all these analyses use the same source, or are there independent estimates that converge? But anyway, let’s assume that 16 TW is in the ballpark. How much electricity does California use? (Recall that this was the quantity of energy Guy Sorman thinks would require a California-sized solar panel to produce.) A couple of sources say that California uses about a quarter million gigawatt hours of electricity in a year. How much is that? WA provides useful comparisons:

a quarter million gigawatt hours = 1/15 total production of electrical energy in US in 2001 = 2 * electricity consumption of Norway in 1998 = 28 * daily electric energy production of all nuclear power plants

a quarter million gigawatt hours / 1 year 2000 hours = 28.5 125 GW

45 TW is about 1600 360 times more than that, so I think that’s how wrong Sorman’s claim is.

If a California-sized solar panel could produce 45 TW, then how much of California would be needed to produce 28.5 125 GW? Let’s ask:

area of California / 1600 360 = 102 455 square miles = 1/15 1/3 area of Rhode Island = 2.2 * area of Walt Disney World 1.1 * land area of Hong Kong

As with other essays in my energy literacy series I am mainly interested here in how tools like WolframAlpha can help us reason our way through a thicket of quantities and unit conversions. Hundreds of people liked Guy Sorman’s essay on Facebook. Dozens tweeted it and the one comment I can retrieve calls it a “balanced and scientific assessment of world’s current and future energy outlook.”

I don’t know enough to agree nor disagree with Guy Sorman’s conclusion:

Future energy supplies will most likely rely more and more on miniaturized nuclear plants and shale gas — a mix capable of responding to a rapidly urbanizing world population’s growing demand for electricity.

But I do know that we need to be able to verify and reproduce analyses that purport to be scientific. WolframAlpha has become the View Source button that helps me do that.

This National Geographic video about 3D printing exemplifies the worst kind of gee-whiz reporting. Just scan a crescent wrench, print it, and bingo, you’ve copied a real tool with moving parts!

Not.

A commenter notes differences between the copy and the original and concludes:

If the real wrench was simply scanned, this would not have happened. A human has built the design data.

3d printing is cool, why do they feel they have to lie about the input method?

The input method is, of course, 3D CAD. From the product brochure for the printer:

Z Corp.’s 3D printing technology leverages 3D source data, which often takes the form of computer-aided design (CAD) models.

Gee-whiz reporting insults our intelligence and trivializes its subject matter. It’s fun to imagine a magic replicator, but it’s more interesting to know about the human/computer interaction that makes real replication possible.

Once I wrote a review of a dozen 3D CAD programs for BYTE Magazine. The benchmark was a model that I commissioned an architect to design. We called it the BYTE Pantheon and it looked like this:

My job was to construct that model in each of the dozen CAD programs. It was hard! That was partly because I had no prior experience with CAD software. But it was also because each program had its own way of using 2D gestures to manipulate 3D objects. That was my main takeaway from the project. There wasn’t (and I think still isn’t) a standard suite of gestures. Even if there were, and even (I suspect) when you can use 3D gestures, it would still be hard because you are making a precise description of a complex object. Wikipedia calls CAD an “industrial art” for good reason. Models with the same functional qualities can differ in terms of style and sophistication. Those differences come into play when the model is shared and modified. Or so I imagine, anyway. I’m not a 3D modeler but I understand 3D modeling to be a process akin to programming.

A friend of mine, Gary Spykman, describes himself as a designer, furniture maker, and artisan. He has also become a 3D modeler, and uses SketchUp to explore his designs and render them for clients. A couple of years ago, Gary designed and built what he calls his cabana. Here it is as designed in SketchUp, and as built in Gary’s back yard.

When I saw what Gary was doing in SketchUp I was inspired to try using the program for some simple needs of my own — to visualize my geodesic tomato suspension dome, and more ambitiously to visualize a remodeling of our kitchen. And you know what? It was just as hard as I remembered! If you do this stuff for a living, as Gary does, then it becomes second nature. But if you only do it occasionally, like me, you’ll be impressed every time with the level of skill required to precisely describe an object or a scene.

Like the commenter on YouTube I have to ask: why lie about this? National Geographic’s gee-whiz reporting doesn’t just fail to inform. It also fails to celebrate the synergy between computational power and human skill that makes 3D modeling so fascinating.

Last week I made my own modest contribution to the growing literature on how to prepare for an Ignite talk. Now the talk is up on YouTube. It’s about barefoot running and whether we really need an app for that.

Recently I did my first Ignite-style talk[1]. It’s an interesting format: a 5-minute 20-frame slideshow set to auto-advance every 15 seconds. The format has its roots in the 5-minute lightning talks that I remember from early Perl conferences. In lightning talks the slides were optional, you just had to finish on time or get gonged. (I can still see Larry Wall cheerfully ringing the gong on others and just as cheerfully having it rung on him.) Ignite makes slides mandatory. The 15-second cadence invites you to think in stanzas; it’s a nice constraint to embrace.

The lore on how to prepare for these talks also has roots that connect Mark Fowler’s 2004 Giving Lightning talks to Jason Grigsby’s 2008 How to Give a Successful Ignite Presentation to many others more recently. Everyone agrees: practice. But how?

For mine I wrote a script in 20 stanzas and then set about tuning them to the required intervals. My first thought was to refer to a clock while reading the script aloud and editing it, but it was hard for me to look back and forth between the clock and the script. So I made a couple of audio tracks for timing. countdown-slide.mp3 says “3, 2, 1, slide” and then every fifteen seconds, “slide” again, finishing with “end.” countdown-5-10-slide.mp3 adds “five” and “10″ at the appropriate intervals. Both turned out to be helpful in different ways.

For tuning the written script to the intervals, I used the countdown-5-10-slide track which gave me plenty of cues to help gauge the edits. For practicing the script I used the countdown-slide track which emits exactly the stream of cues you get in the talk: start, a cue every 15 seconds, then stop.

As is often appropriate for an Ignite talk my slides were mainly pictures not words. I tried to search for and use only images licensed for sharing, but the discoverable pool of such images isn’t nearly broad or deep enough. So I fell back to grabbing images from Google and Bing searches, feeling guilty about that, and wondering what it will take to make the pool a lot broader and deeper.

I wasn’t sure at first I’d need the countdown-slide track — the one that just says “slide” every 15 seconds. If you practice and memorize the script while watching the slides then you’ve got your cues, no need for audio timing. But after a few run-throughs I got antsy and wanted to go for a hike. That’s where the countdown-slide track really worked, in a couple of ways. It not only provided the cues, it prompted me to visualize the slides. From then on I could practice anywhere I wouldn’t mind being seen talking to myself: running, hiking, driving to the airport. I hardly used the slides again.

When I did use the slides, in a few practice runs and then in the talk, I saw how helpful it had been to have visualized them, but not seen them, while practicing. I’d learned to do the talk from memory without the slides. Doing it with them was, by comparision, easier.

This makes perfect sense, of course. I think it’s related to how musicians memorize music: hear the tune, see the notes on the page, feel your fingers on the instrument. Then selectively omit the audio track, the sheet music, and even the instrument, and do the hearing, seeing, and feeling in your mind.

A couple of weeks later I was on a panel where I had up to 10 minutes to speak. I wound up using only 4 minutes, and while there weren’t any slides, I still wrote it out as a story told in memorizable stanzas. I think it turned out better than if I’d used the whole time for something less tightly constructed.

The 4-minute panel talk turned out to be harder to learn than the 5-minute Ignite talk, and now I see why. Even though slides weren’t required, I could have used them as practice cues! I guess that’s what Joshua Foer and other memory experts keep telling us: divide things into chunks, tag the chunks with pictures.

[1] The subject was barefoot running. I think it’ll be posted soon.

Yesterday I joined a panel at the New England Conference of Public Utilities Commissioners. I was the odd man out in the group but that was the point. The organizers wanted somebody other than the Usual Suspects to bring an outside perspective to the panel. Here’s roughly what I said.

My phone bill is itemized down to the last minute and second — every call, every number. For the longest time I’ve wanted my electric bill to be itemized in the same way, right down to the watts actually used by every appliance.

How can you decide whether to replace your old fridge if you don’t know what it really costs to run it? Or how guilty should you feel, or not feel, about turning on the air conditioner in June, if you don’t have the data?

Well I’ve finally got the data, at least sort of, thanks to the smartmeter I installed recently. It’s a first-generation device; it’s a little flaky; it doesn’t track individual appliances. But it does give me realtime feedback about how many watts my house is burning.

That’s a really interesting number. When you turn off some lights, you see the difference right away, in watts and in pennies per hour. It’s a powerful behavior changer.

Around the time I was installing this thing I heard a commentary on NPR about smartmeters, The title of the piece was “Smart Meter, Big Brother,” and you can guess what it was about. Smartmeters are an unpredictable new technology, they bring unforeseen privacy risks that none of our statutes and regulations have ever thought about.

What was the guy worried about? Well, suppose you come home every night around the time the bars close. The electric company can tell because it sees your lights and appliances come on. Then somehow it leaks that data to your insurance company, which raises your premiums.

Really? I don’t know, to me there’s nothing new here. Most people I talk to have given up on privacy. They just expect that’s what would happen with your data.

Here’s what would be a new twist. Why do we just assume that a smartmeter will automatically phone its data home to the electric company? Mine doesn’t. It only feeds data to my own home computer, and from there to wherever I route it. PSNH doesn’t know anything about this. [ed: Well, I guess they do now :-)]

This reminds me of what we’ve been seeing in IT for quite a while. Consumer technologies at the edge of the network have disrupted enterprise technologies at the core, Telecom feels this disruption intensely right now. I’m wondering if other utilities, like power, will start to feel it too.

I know a guy who runs a company that does demand management for big box retailers like Michaels and Petco. He drops a package of instrumentation and controls into your store, he connects it to the web, and then when there’s a rolling brownout in California he can dial down the lights and ventilation and AC in all the buildings across his network.

He picked the retail sector because every one of those stores has the energy footprint of 20 or 30 homes. And he avoided the residential sector because it’d be a lot harder to have the same kind of impact there.

But now I’m wondering if we’re going to start to see the crowdsourcing of demand management. I’ve already got a do-it-yourself Internet-connected smartmeter. How much longer until I can add DIY controls to dim the lights or schedule the dishwasher to run off peak? And then how much longer before I can connect up with a bunch of other people who want to do the same things?

I’m not saying this will happen. But it could. And if it did, we wouldn’t need to wait for legislators and regulators to figure out how to deal with some new privacy threat, because there wouldn’t be a threat. It would just be people choosing to share their data with other people in order to conserve energy. And, by the way, they’d be having a lot of fun doing it too! Think massively multiplayer online game for the smart grid.

Now maybe I’ve told the wrong crowd about my DIY smartmeter. Maybe I’ve already broken some rule I don’t know about. But if not, or in any case, I’d like to put this crazy idea onto the table for discussion.

For me it was a chance to hang out with a bunch of public utility regulators and watch them wrestle with thorny issues. Does regulation often stifle innovation? Yes. Is regulation a means to socially just ends, like rural broadband? Yes. I found it fascinating.

Also, I got to spend an evening and a morning at the fabulous Mount Washington hotel during a perfect New Hampshire solstice.

I guess because I lost my own dad a few years ago, I was really moved by Sunday’s outpouring of Father’s Day sentiment. On Twitter, on Facebook, on blogs, everywhere you looked online you found people sharing an “I miss my dad” moment. Then today, while searching YouTube for renditions of a beautiful Hawaiian slack key guitar tune I’m learning, called Moe ‘Uhane, I found this wonderful version. And even better, this comment:

I was there when he taped this on my little tape recorder back in the 80′s. I was sleeping and woke up to find him playing still in his pj’s. Miss my dad. Thanks for representing. Sounds awesome!

- Mahina Chillingworth

Sweet! The tune came to Sonny Chillingworth in a dream, and then his dreaming daughter woke up to hear him recording it.

Ernest Hebert is an author and Dartmouth professor who was born in Keene and writes fiction about this area. During an interview with the Keene Sentinel he unloaded a gorgeous rant about William Faulkner. It’ll soon vanish behind the Sentinel’s paywall. So as a public service, and to ensure I can refer to it later, I’m placing it here for safekeeping.

Q: How do you feel about being compared to William Faulkner?

A: I hate being compared to Faulkner — this kind of uppity, snooty southerner with his turgid prose based more or less on the Bible. I can’t bear to read Faulkner. It makes me want to puke. I just loathe Faulkner. And you can quote me on all of that.

Done!