Jon Udell

On certain summer nights in Keene, NH, you can wind the clock back 100 years and experience baseball from another era. Our town hosts an NECBL (New England College Baseball League) team called the Swamp Bats. Their home games, at the high school’s Alumni Field, are like Norman Rockwell paintings come to life. Fans arrive early to set up lawn chairs along the third base line. Between innings children compete in egg-balancing races. A few of the players will end up in the big show, but all of them remind us why baseball is the national pastime.

We like to get out to at least a few games each year. In years past the Bats’ Google Calendar helped remind me to do that. But this year the schedule became a silo. The data is trapped in a web page. It can no longer flow, as it once did, to Keene’s calendar hub or to my personal calendar.

Happily I found a way to liberate that data. So now you can once again see the Swamp Bats calendar here and subscribe to it here. The circumstances and techniques that made this possible are worth exploring.

It turns out that while the Swamp Bats’ own calendar is a data silo, the NECBL’s master calendar does the right thing and complements its HTML view with an iCalendar feed. Crucially the NECBL’s calendar does another right thing: it adopts a consistent naming convention. The titles of events on the calendar look like this:

VM@KS 6:30pm

DW@SM 6:30pm

NB@NG 6:30pm

That’s shorthand for:

Vermont Mountaineers vs Keene Swamp Bats at Keene

Danbury Westerners vs Sanford Mainers at Sanford

New Bedford Bay Sox vs Newport Gulls at Newport

To create an iCalendar feed for Keene home games, I used an iCalendar filter to find “@KS” in the titles of the NECBL feed and produce a new feed with just the matching events.

Since I’ve recently started an elmcity hub for Montpelier Vermont I did the same thing there. Filtering on “@VM” gives a view for the Vermont Mountaineers’ home games that you can see here and subscribe to here.

These maneuvers suggest an interesting twist on the free rider problem. When one person upgrades the quality of an online information source and makes the improved source available to everyone, the free ride everyone can enjoy isn’t a problem at all. It’s a solution.

Last week, for example, I visited Montpelier to speak with a state official about the elmcity project. He described a problem that’s familiar to many parents. His kids are in a sports league; their schedule comes to him formatted as an Excel spreadsheet; he can’t merge that Excel schedule with his personal calendar. So he transcribed the data into Google Calendar. And having done so, he shared it with other parents. That’s a free ride the sports league ought to provide. But in this era of abundant free cloud services, any parent can offer it instead, merely by satisfying a personal need. Web users who become web makers can turn the free rider problem into a solution.

My recent column on smart meters came to the attention of Richard Stallman, who worries about the privacy and surveillance issues I alluded to. In the course of our email discussion a question came up that I’d like to answer but can’t. When a smart meter is utility-owned, rather than DIY like mine, do any of the providers offer choice with respect to the granularity of the data feed that’s phoned home? In theory it would be possible to opt out of a realtime feed and only use the meter to automate the monthly accounting that’s currently still done by a visiting person. I doubt that any utility offers that but I’d like to be proved wrong, and either way it would be nice to know for sure.

Richard, by the way, would like to add the privacy/surveillance issues arising from smart meter deployment to his list of causes, and is looking for someone who wants to lead that charge. If you’re interested and qualified you are welcome to contact him. Here, as I have now learned firsthand, is the protocol. You’ll write to him at rms@gnu.org. You’ll receive an autoreply that begins:

I am not on vacation, but I am at the end of a long time delay. I am located somewhere on Earth, but as far as responding to email is concerned, I appear to be well outside the solar system.

After your message arrives at gnu.org, I will collect it in my next batch of incoming mail, some time within the following 24 hours. I will spend much of the following day reading that batch of mail and will come across your message at some point. If I write a response immediately, it will go out in the next outgoing batch–typically around 24 hours after I collected your message, but occasionally sooner or later than that. Please expect a minimum delay of between 24 and 48 hours in receiving a response to your mail to me.

If a conversation ensues, it will happen on that cycle. This strikes most people as odd. As Jeremy Zawodny once noted, though, Richard Stallman is Getting Things Done. Old School.

A couple of weeks ago all the posts here became invisible. There didn’t seem to be anything I could have done wrong to cause that, so I wrote to the support team at WordPress.com about it. I got a prompt acknowledgement from Erica V. that something was, indeed, wrong. Soon after that she confirmed that the problem was fixed. That left me feeling pretty good about WordPress.com. It’s a free service, after all, I’m only a customer in a rather minimal way: I pay for domain name redirection and for the ability to edit my CSS file. Yet the customer service I received was outstanding.

Then, last week, something else went wrong. The widgets in the right column were getting bumped down by a post that didn’t belong in that column. I tried a few debugging strategies and then wrote to WordPress.com support again. Here was the prompt response from macmanx:

You’re all fixed up now. You had an extra div tag in “Meta-tools for exploring explanations,” but I removed it.

Oops. I know how to validate HTML and should have caught that myself. It’s not something I’d have wanted to bother the support team with. But they didn’t make me feel like a jerk. Again the problem was handled promptly and cheerfully.

You know, we tip for all sorts of services in the physical world, including ones delivered far less capably. Why not tip for excellent online customer service? If there were a tip jar for WordPress.com I’d have used it both times.

Yesterday’s post contains an error so embarrassing that I was briefly tempted to yank the whole thing. But of course That Would Be Wrong. What’s more, the error supports the larger point I was trying to make before I derailed myself.

I was talking about Bret Victor’s notion of explorable explanations, which he illustrates on a page called Ten Brighter Ideas. I’d looked at it before, but when I revisited it yesterday I had trouble believing that the following claim could be true:

If every US household replaced 1 incandescent bulb with a compact fluorescent bulb, we’d save 11.6 TWh (terawatt hours), which is the energy equivalent of 1.5 nuclear reactors or 9.5 coal plants.

Some people intuit what these units and quantities mean. But most of us — me included — don’t. And even experts are prone to error. A few months ago I spotted one such error. A Ph.D. economist wrote an editorial that consistently used billions of barrels of oil rather than, as intended, millions. The column was syndicated to hundreds of newspapers, and so far as I know nobody noticed until I happened to check.

What prompted me to check? My friend Mike Caulfield, who’s been teaching and writing about quantitative literacy, says it’s because in this case I did have some touchstone facts parked in my head, including the number 10 million (roughly) for barrels of oil imported daily to the US.

The reason I’ve been working through a bunch of WolframAlpha exercises lately is that I know I don’t have those touchstones in other areas, and want to develop them. Having worked a few examples about global energy, I thought I’d built up some intuition in that realm. But in this case the intuition that prompted me to check Ten Brigher Ideas was wrong.

When I did check, things went completely off the rails:

If 111 million households each swap out one 75W bulb for a 25W bulb, saving 50W each for 180 hours (i.e. half of each day for a year), we’re looking at 100,000,000 * 50W * 180hr = 999GWh. We’re off by a factor of about 1000.

As Pasi points out in a comment:

Hmm, “half of each day for a year” is not 180 hours, but 365*24/2=4380 hours?

My brain thought days, my fingers wrote hours. I think I’m slightly dsylexic when it comes to units, and so I’m prone to that sort of error. It’s another reason why I use WolframAlpha to check myself. When I do that, I try to take advantage of WolframAlpha’s marvelous ability to automate conversions. For example, during an earlier exercise I needed to visualize the gallon equivalent of the energy released by combustion of one kilogram of gasoline. Normally this would entail looking up the density of gasoline, 0.726 g/cm³, applying that constant, and then converting to gallons. But in WolframAlpha the phrase density of gasoline is meaningful and can be used directly, like so:

http://www.wolframalpha.com/input/?i = ( 1 kilogram / density of gasoline ) in gallons

Similarly, here’s what I could have done to check the Ten Brighter Ideas claim:

http://www.wolframalpha.com/input/?i = (1/2 year) * 111,000,000 * 50W as TWh

That comes to 24 TWh, which is in the ballpark of the claimed 11.6. Maybe Bret assumed lights are cumulatively on 1/4 of the time, I haven’t checked, but if so that would nail it.

Why didn’t I write the WolframAlpha query that way in the first place? Because, I think, we still expect to do a lot of basic computation ourselves. You want the answer in hours? Put hours in. How many? You can figure that out. But should you?

I think it depends. It’s good to exercise your inboard computer — not only to calculate results but also to store and retrieve certain touchstone values. But it’s also good to delegate calculation, storage, and retrieval to outboard computers that can do these things better than we can — if that delegation can be smooth. WolframAlpha points to one way that can happen, Bret Victor’s simulations point to another.

At the Canadian University Software Engineering Conference in January, Bret Victor gave a brilliant presentation that continues to resonate in the technical community. No programmer could fail to be inspired by Bret’s vision, which he compellingly demonstrated, of a system that makes software abstractions visual, concrete, and directly manipulable. Among the inspired were Eric Maupin and Chris Granger, both of whom quickly came up with their own implementations — in C# and ClojureScript — of the ideas Bret Victor had fleshed out in JavaScript.

Others imagined what the educational value of such a system might, or might not, be. Noting that Bret’s demo “looks like a powerful visualization system,” Mark Guzdial wrote:

The problem is that visualization is about making information immediate and accessible, but learning is about changes in the mind — invisible associations and structures.

Ned Gulley echoed Mark’s concerns. Citing Bret’s 2009 essay Simulation as a Practical Tool — which contains live demos that use simulation to solve a physics word problem — Ned writes:

It’s beautiful! And it works well as long as you don’t want to modify the essential parameters of the problem. But Victor isn’t helping us learn the metatools that he uses to create this environment. To be fair, that wasn’t his goal, but as a user, I feel like I’m locked in a pretty small task box. More to the point, it’s expensive to create these interactive gems, and there’s only one Bret Victor.

Victor’s real power is his ability to rapidly create and deploy these tools. In a twinkling he can size up a task that is worth studying, put a box around it and spin a tool. He does this so effortlessly, with such mesmerizing legerdemain, that we lose sight of this meta-skill. What Victor was really doing in his talk was illustrating the power of tool spinning, the rapid creation of customized, context-sensitive, insight-generating tools. Direct manipulation is good, but the nature of direct manipulation changes with the context, and the context can’t always be anticipated.

My preferred goal is to make tool spinning (and tool sharing) as easy as possible. If tool spinning is easy, if that is the expressive skill that we give our users, then small task boxes aren’t a problem. You can always make more tools.

Don’t use the thing Bret made. Do the thing that Bret does.

Ned’s right about tool-spinning and I’ll come back to that. But first I want to consider Mark Guzdial’s critique, cited by Ned, that interactive simulations don’t help us build the mental muscles we need to reason analytically and symbolically. Indeed not. But Bret doesn’t claim that they do! In Simulation as a Practical Tool he says flatly:

Regardless of how it’s taught, most people will never be comfortable entering this level of abstraction in order to explore the problems of their lives.

That’s true for the physics problem Bret solves in that essay, it’s true for the programming challenges he tackles in his CUSEC demo, and it’s also true in all scientific and technological realms. Some people can reason effectively using abstract symbol systems. Most think more concretely and reason best about what they can see, touch, explore, control. What matters in the end is that people can reason effectively, about all sorts of things, especially about the scientific knowledge and technological machinery on which our civilization depends.

In his 2011 essay Explorable Explanations, and the companion demo Ten Brighter Ideas, Bret imagines a world in which public discourse about energy (or health or climate or economics) is facilitated by an explorable explanation that:

• States assumptions, and ties them to supporting data.
• Makes connections among interacting variables.
• Enables the viewer/user to vary assumptions and observe consequences.

An example from Ten Brighter Ideas:

Suppose 100% of US households replaced 1 bulb at random with a compact fluorescent.

This, we learn, would save 11.6TWh per year, which is equivalent to 1.5 nuclear reactors or 9.5 coal plants. The underlined elements — 100% of households, 1 bulb — become sliders you can use to vary these inputs. What if only 20% of households get with the program? If each replaces 5 bulbs, we can get the same 11.6TWh savings.

Wince. (See comments.)

We need robust explorable explanations that state assumptions, link to supporting data, and assemble context that enables us to cross-check assumptions and evaluate consequences.

And we need them everywhere, for everything. Consider, for example, the current debate about fracking. We’re having this conversation because, as Daniel Yergin explains in The Quest, a natural gas revolution has gotten underway pretty recently. There’s a lot of more of it available than was thought, particularly in North America, and we can recover it and burn it a lot more cleanly than the coal that generates so much of our electric power. Are there tradeoffs? Of course, There are always tradeoffs. What cripples us is our inability to evaluate them. We isolate every issue, and then polarize it. Economist Ed Dolan writes

These anti-frackers have a simple solution: ban it.

The pro-frackers, too, have a simple solution: get the government out of the way and drill baby, drill.

The environmental impacts of fracking are a real problem, but one to which neither prohibition nor laissez faire seems a sensible solution. Instead, we should look toward mitigation of impacts using economic tools that have been applied successfully in the case of other environmental harms.

In order to do that, we’ve got to be able to put people in both camps in front of an explorable explanation with a slider that varies how much natural gas we choose to produce, linked to other sliders that vary what we pay, in dollars, lives, and environmental impact, not only for fracking but also for coal production and use, for Middle East wars, and so on.

Where will these tools come from? As Ned Gulley says, they’re expensive to build, and you need a scarce talent Bret Victor to build them. And even then, as we’ve seen here, it’s tricky to get right. Where will the tool-spinning meta-tools come from?

I actually think Bret knows, or at least intuits very well, how we can get there, and I think he has shown the way in his book-length 2006 essay MagicInk. That’s what I meant to write about today, but after an unexpected detour through Ten Brighter Ideas I think I’ll do that another time. Meanwhile if you haven’t read MagicInk I commend it to you.

In Fluid Concepts and Creative Analogies, Douglas Hofstadter (and a crew of talented students) argue that analogy-making is a core characteristic of human intelligence. The book is full of delightful puzzles. One class of puzzle goes like this:

• What is the Ob of Nebraska? (The Platte. Hence the name for these: Ob-Platte puzzles)

• What is the Newark of Delaware?

• What is the Gettysburg of Hawaii?

The authors write:

If one were to ask, “What is the Atlantic City of France?”, a large number of people would undoubtedly answer “Monaco”, even knowing full well that Monaco is not a city inside France, but rather an independent, albeit small, country that is not in France, but borders it (the gambling city within Monaco is Monte Carlo). However, Monaco is world-famous for its casinos, is small enough to be thought of as a city, and like Atlantic City, is located on the sea.

I’m working on an Ob-Platte puzzle that you may be able to me with. It goes like this:

Who is the Andy Morikawa of Ann Arbor?

I’m working on the same puzzle for Berkeley, Boston, Providence, Portland, Seattle, Toronto, and a few other places where I’d like to convene a seminar on community information management. The first of these was held in Blacksburg, Virgina, during my recent visit there, and was successful because Andy Morikawa knew and invited all the right people. The Andy Morikawa of Blacksburg is a fellow at Virginia Tech’s Institute for Policy and Governance, and was until recently the executive director of the Community Foundation of the New River Valley.

Just as Monaco is not actually a city in France, the Andy Morikawa of Ann Arbor, Berkeley, and Providence might or might not be affiliated with the universities in those towns. Another Andy might be an independent community organizer, or a city councilor, or an economic development specialist, or the director of a nonprofit chartered to consolidate and promote cultural arts. He or she will certainly, though, be a well-known connector who has the ability to convene the kinds of folks I want to reach. They’ll come from the public schools, the colleges, the city government, the newspapers and online media, the music and dance scenes, the public library, the hospital, the chamber of commerce, the downtown merchants association, arts and culture nonprofits, social services organizations, sports leagues, environmental groups, and other centers of public life.

Crucially I am not looking for webmasters and IT staffers, but rather for leaders responsible for public engagement. The principles I’ll explain and demonstrate are ones that everyone can understand and apply. Once leaders understand what’s possible they can delegate to staff, but what they’ll delegate is nothing they couldn’t pretty easily do for themselves.

What makes this a tricky problem in social networking is that my best contacts are in the technical world, but I’m trying to reach outside it. That’s why I need an Andy to help me bridge the gap. If you are an Andy, or if you know an Andy, I’d really like to hear from you.

The bedrock principle at the core of the elmcity project is: own your data, syndicate it into contexts that need it. This applies far more broadly than to calendars, but I’m focusing on calendars now because we all still struggle to make sense of various personal, professional, and public timelines, so everyone (I hope) can relate to the challenge and the opportunity.

For the sorts of calendars you see at elmcity.cloudapp.net, which represent schools, libraries, museums, music venues, sports clubs, and other organizations offering schedules of activities to which the public is invited, the application of the principle is as follows. Publish your calendar from a single canonical source as two streams of information. One is an HTML page for people to read and interact with on your own website. The other is an iCalendar (ICS) feed that can syndicate to other sites.

One popular way to publicize events, Facebook, doesn’t fit neatly into this model. For some organizations, a calendar on a Facebook page is the canonical source. I respect the choice to use Facebook in that way, and support it by enabling such calendars to syndicate from Facebook into other contexts. But I don’t recommend the choice.

Consider this link that appears on the Peterborough NH timeline for April 28:

Sat 09:00 AM Childhood Pop-Up Museum Peterborough Historical Society Museum and Archives

It’s nice that this event is able to show up in a public timeline for the Monadnock region, along with events from many other sources. But if you click the link … well, it depends. If you’re logged into Facebook you’ll see the details. Otherwise not.

The Peterborough Historical Society does publish the event on its own site, here it is, but there’s no iCalendar feed on the site. The only machine-readable calendar for the Peterborough Historical Society available on the web is the Society’s Facebook page. And while that page is on the web, it’s only fully available to the subset of the web formed by logged-in Facebook users.

What I want, and what I think everyone who promotes public events should want, is the inverse of that arrangement. I think we should put our public calendars on our own sites. And then, since Facebook is a wonderful tool for socializing our calendars — that is, discussing upcoming events, keeping track of who’s planning to attend — we ought to be able to syndicate from our own calendars into Facebook.

It’s doable, and I’ve prototyped a solution, but in light of an upcoming change in the way Facebook handles API access tokens I’m going to wait for the dust to settle before rolling it out. I’m told there may soon be a new mechanism that will simplify the authentication dance the elmcity service will need to perform in order to make this happen.

Meanwhile, though, I’ve implemented a variant of the idea. On Saturday a favorite local band, Jatoba, will be playing here in Keene at Waxy O’Connor’s. I found the event on the MonadnockNH music calendar, to which it had syndicated from two sources: Eventful.com and Waxy’s own calendar. I’m hoping some friends will want to join me there, and Facebook is a great way to alert them that I’m going. So I posted the event to my Facebook page.

Then I realized it was silly for me, or anyone, to have to retype data that’s available in machine-readable form. So I’ve expanded the elmcity service’s Add to calendar mechanism. Options were formerly Add to Google Calendar, Add to Hotmail Calendar, and Add to iCal (for generic iCalendar clients). Now there’s also Add to Facebook. When you see an event on an elmcity timeline that you’d like to socialize, it’s now just two clicks to pluck it from the timeline and announce it to your Facebook friends. It’s kinda cool, and it’s a nice warmup for the wholesale syndication I’m planning to do later.

Ownership of data and services based on data are separable concerns, and it’s crucial that we begin learning how to separate them. In this case the owner of the data about Jatoba’s appearance at Waxy O’Connor’s is Waxy O’Connor’s.

(An aside: You might argue that the owner of that data is really Jatoba, and that Waxy O’Connor’s could be syndicating from Jatoba’s site. That’s true. You might also argue that Waxy O’Connor’s doesn’t really own its calendar, Google does. That’s true too. Waxy O’Connor’s has no more of a contractual relationship with Google than it might with Facebook. But Waxy O’Connor’s does have a contractual relationship with its site provider, and that contract could include provision of a calendar service that puts Waxy O’Connor’s in charge of its own data. Likewise for Jatoba.)

(Another aside: Eventful and Waxy’s do not agree on the start time for Jatoba’s appearance. This is not uncommon. Eventful says 8PM, Waxy’s says 9PM. I’m going at 9PM because I regard Waxy’s as the more authoritative source in this case. I’m not sure where Eventful got its information, but if it were syndicating from Waxy’s or from Jatoba then we wouldn’t see this temporal drift.)

What Facebook can bring to the table, if we look at it the right way, is a service based on data owned by Waxy O’Connor’s — or by Jatoba. It’s the service, again, of socialization: announcing an event to a group, discussing the event with the group. Music promotion is interesting in that there are three ways that socialization might happen. A band might want to do it, and/or a venue, and/or a fan.

So, for example, an event that was syndicated from Waxy O’Connor’s site (or from Jatoba’s site, or from both), might want to be injected into a Waxy O’Connor’s Facebook Page, and/or a Jatoba Facebook Page, and/or the personal Facebook page of a fan like me. The latter option is available now. The former ones are coming soon.

If you keep a calendar on a Facebook Page, and wish to regard it as the authoritative source for the events on that calendar, then you can use the elmcity service to synthesize an iCalendar feed from that Facebook Page for use elsewhere. But what if you want things the other way around? That’s the question that came up during Saturday’s workshop¹. Several folks asked whether calendars they maintain authoritatively elsewhere can syndicate into Facebook. It’s something I’ve been meaning to enable, so I looked into it today.

I’ve mocked up a simple mechanism that you’d use to identify an iCalendar feed and a Facebook Page that you control, and then tell the elmcity service it’s allowed to inject events from the iCalendar feed into your Facebook Page. My notion was that you’d only need to use this mechanism once, to hook things up. Thereafter the elmcity service would sync the iCalendar feed to the Facebook Page on some regular schedule.

In order to do that, though, the elmcity service would need to hold onto the access token that it got from Facebook. The needed permissions are: manage_pages (so the elmcity service can update the Facebook Page), create_event (so it can create events on the Facebook Page), and offline_access (so the elmcity service can keep updating the Facebook Page without asking you). The last of these, offline_access, is a very powerful privilege, arguably too powerful, and sure enough it’s going away in a few weeks.

What I think this means is that it will still be possible to sync from an iCalendar feed to a Facebook Page, but that the owner of the Facebook Page will need to be notified periodically (every 60 days, it seems) to reauthorize the service that does the synchronization. Have I got that right? If so it’ll be awkward, but I won’t really be able to complain. I agree that users should periodically review the powers they’ve granted to third-party apps and services.

On Thursday I visited Gardner Campbell’s class. For me it was a chance to try to convey, to students who are mainly not on the engineering track, some key intuitions that arise from the architecture of the web. At one point I invited them to summon a grain of sand into existence, name it uniquely, cast it onto the web, and wait for search engines to notice.

At least one student, lisskane, tried the experiment. She wrote:

I’m going to completely admit that before this class I knew absolutely nothing about the WWW and Internet infrastructure. And honestly, it still confuses me. Who knew that the web and the Internet were different things? I thought they were simply interchangeable words, synonyms that could be looked up on Microsoft word. But they are different and when to my surprise when I click on Internet, there are no synonyms for it. It is it’s own thing. Which brings us to what Jon Udell was saying in class. When it comes to the World Wide Web, our imagination is our only limit. We decide what we want to make. Who we want to share it with and what we want it to mean. Take words for example.  Dackolupatoni: when you Google that, nothing comes up. But here is our experiment. Let’s search it in a few hours and probably this will come up. Which is recursion. And then that person can be linked to this. And it goes on and on. Now isn’t that cool?

It sure is! Unfortunately for lisskane her grain of sand, dackolupatoni, has not yet fully awakened. Bing finds a tweet I posted yesterday. But neither Bing nor Google, as of Saturday morning, has yet made lisskane’s experiment in discovery and recursion come alive. This pains me. I want her to see it happen. But I realize I forgot to mention something. This power to cast grains of sand into the virtual world and than awaken them is a power that grows as you exercise it. Mine, after many years, is strong. Hers is nascent. But thanks to another wonderful property of the web I can share some of my power with her. So, Google and Bing, please awaken lisskane’s grain of sand.

Today I met a university instructor who works in the area of materials science and engineering. She is also a guitarist and a maker of guitars. These interests combine in the following way. Her students work on projects that involve, for example, curing of finishes on instrument-grade wood. Then they travel to the factory of a leading guitar manufacturer where they present their findings to engineers and designers. During their on-site visit they stay not at a hotel, but a rented vacation home where the bond that forms among students lucky enough to have this experience is strengthened by the shared living arrangement.

The university knows about this activity, it’s been written up on the website. What the university doesn’t know, it seems, is that this instructor has been unable to land any funding to support these trips. For five years she and a collaborator from another school have paid for travel and lodging — their own and the students’ — out of their own pockets. Which, in her case, is more than a bit of a stretch.

What do you do when you hear a story like this?

• Rail against a system that does not recognize or support some of its truest heroes.
• Point her to Kickstarter.
• Celebrate the human spirit.
• Give her a hug.

I’m going to skip a) because life’s too short and it’s boring. b) Done. c) Done. d) Flubbed my chance but I hope to fix that.

In A tale of two dams I proposed using a tag, WestStDamKeene, to coordinate public discourse about a decision we need to make here in Keene, NH. Should we repair the Ashuelot River Dam on West Street, or remove it?

On the day I wrote that post, it was the only document on the web containing the 14-character string WestStDamKeene. A few months later I mentioned it again in Awakened grains of sand¹, which considers what we can do when we invent names that are unique to the web — and that can be found by searching for them.
So now a Google or Bing search would find two documents on the web containing the tag WestStDamKeene.

Yesterday I tried that again and was delighted to find a number of results, notably from two local organizations deeply involved in the decision about the dam: the City of Keene [Google, Bing] and the Keene Sentinel [Google, Bing].

Among the links that turn up in a general search for WestStDamKeene is this pointer to minute 40 of the two-hour city council meeting on Oct 6, 2011, when discussion of agenda item F begins. That happens because the title of item F is Ashuelot River (West Street) Dam (onlinetag: weststdamkeene) – Department of Public Works.

This makes me very happy! The City explains:

The City of Keene has embarked on a new online initiative to streamline information sharing with the citizens of Keene. The City will be “tagging” certain projects or initiatives that have generated a significant amount of community interest. This initiative will be a collaborative effort with the Keene Sentinel, which will be applying the same tag to their articles. The tag assigned by the City can be used to search all online content associated with the item through the search engine of your choice.

And it adds:

As you collaborate with others online to discuss City projects, you are welcome to tag your posts, emails, articles, etc. with the same tags. For example, if you belong to a Keene neighborhood group, and you have information to share regarding the West Street Dam project on the group’s Facebook page, make sure to include “weststdamkeene” in your post. Your post will then appear in a Web search alongside City and Sentinel tagged content.

If others follow that advice and join in, my happiness will increase.

¹ Cliff Gerrish’s poetic response to Awakened Grains of Sand also made me very happy.

In recent years I’ve had increasing trouble with cold fingers. In winter I’ve gone from warmer gloves to mittens to expedition mittens. After even brief exposure to the cold, the circulation in my fingers shuts down and they go scarily white. On winter hikes I carry chemical handwarmers not just for comfort but as an emergency measure, because if anything went wrong and I had to stop moving I’d be highly vulnerable to frostbite.

Finally I learned that the condition has a name: Raynaud’s syndrome. The Mayo Clinic says it’s more a nuisance than a liability, and I guess that’s true, but lately it’s gotten to be a real nuisance. Even indoors, during winter, I’ll realize that my hands are ice cold, and have to run hot water on them in order to resume typing on the keyboard or playing guitar.

I read that some folks have had success with biofeedback so I decided to give it a try. Raynaud’s is just an autonomic nervous response. The shutdown of circulation to hands and feet may be necessary in a survival situation, but with Raynaud’s it happens unnecessarily and inappropriately. The idea of biofeedback is that you can train yourself not to do that.

So I got one of these. It’s nothing special, just a thermometer with a sensor you tape to your finger. You look at the number and try to make it go up.

The first few times I tried nothing happened. The baseline, for me, seems to be room temperature, which in winter might only be mid-60s. I couldn’t budge from that. Then, one day, after about 15 minutes, I gained a tenth of a degree. Then another. Then a degree. Then another. And suddenly the temp was shooting up. In about five minutes it went from 67 to 94.

This wasn’t replicable. Sometimes I’d get that dramatic response, sometimes nothing at all. And it didn’t seem very practical because you can’t do anything else, you have to watch the readout — this is biofeedback, after all.

Eventually I noticed that if I placed my opposite hand — that is, the one without the thermometer — someplace warm, like on my neck, then I could sometimes trigger the warming response in the other hand. So that’s my current best strategy. When I notice my hands feeling cold indoors, I’ll take a “measurement” by placing one hand lightly on the side of my neck. Often I can get both hands to warm up by doing that.

I think the thermometer was useful mainly to prove to myself that the relaxation response is controllable. And that it’s binary, or so it seems to me, by which I mean that for the most part it either happens or it doesn’t. And that some kind of feedback is needed to produce the warming effect I want. Direct self-measurement (i.e. touching my neck) seems to work well enough, and that’s nice because it’s something I can easily do while working.

I’m getting OK results indoors, but still not having much luck outdoors. Now that I know the syndrome isn’t just something that happens to me, though, but rather something I’m doing to myself, I think I might eventually be able to back off from the arctic mittens on days other people are wearing light gloves or nothing.

As I build out calendar hubs for various cities, I’ve begun to develop a scorecard that shows which civic institutions do, or don’t, offer iCalendar feeds that make their public calendars available to their city’s hub. Here’s the scorecard for the current set of featured hubs. It illustrates what I’ve been saying about how the future of community information management is unevenly distributed. If you’re curious about how your city would stack up, it’s easy to find out. Just visit the websites of the institutions shown in the top row, look for the Calendar or Events link, and then look for an iCal link. If you find one for most categories then your city is (at least with respect to public calendars) thinking like the web.

¹ Feeds not easily discoverable, and not valid, but can be made to work.
² Ann Arbor’s public schools are exemplary.
³ Only the Dorchester facility.
⁴ Missing most activities.

It’s been a while since I hung up my spurs as a columnist, and lately I’ve been missing the opportunity to write regularly for a venue other than this blog. So when Mike Barton asked me to contribute to Wired’s Cloudline I said yes. I’m calling the column The Personal Cloud and it’ll run most Fridays. Last week was the introduction, this week I explore what Phil Windley and his crew are doing to help realize Doc Searls’ vision of a Live Web.

The URL for the series is http://www.wired.com/cloudline/tag/the-personal-cloud, and the corresponding RSS feed is http://www.wired.com/cloudline/tag/the-personal-cloud/feed.

Should a tag namespace be a top-down taxonomy or a bottom-up folksonomy? My answer is: both. In recent months, as I curate calendar hubs for selected cities, I’ve been working toward an approach that harmonizes the two styles.

Principle: Top-down and bottom-up

In the elmcity context, the most important taggable object is the calendar feed. That’s because when you can characterize a whole feed with a tag, all the events in that feed inherit the tag. The primary sources of taggable feeds are Eventful, Upcoming, Facebook, Meetup, and EventBrite. I call them taggable because, while some of these services tag individual events, none tag feeds based on venues (Eventful, Upcoming) or Pages (Facebook) or groups (Meetup) or organizers (EventBrite). Assigning feed-level tags is an editorial exercise for the curator.

To these sources I add as many standalone iCalendar feeds as I can find. For Boston and Seattle, the results add up to lists of over 600 tagged iCalendar feeds. Here’s a table of the current list of tags for Boston, the current list for Seattle, and the intersection of the two lists.

Among the dynamics in play here, we can see the general and specific principle at work. For a general tag like university there are city-specific instantiations: bu and northeastern for Boston, uw and seattleu and nscc for Seattle. Likewise for the general tag highschool there are specific tags like boston-latin-hs and cathedral-hs for Boston, garfield-high and ingraham-high for Seattle.

These city-specific tags are top-down in the sense that I, as curator of the hub, have assigned them and made them part of the hub’s core tag vocabulary. But they are also bottom-up in the sense that they represent discoverable sources that are providing enough event flow to warrant such treatment.

These core hub vocabularies are fluid. As I move from hub to hub I’ve been keeping an eye on the common core and refactoring all the hub vocabularies as I go along. I also use these evolving hub vocabularies as templates against which to match vocabularies from other sources.

Mechanism: Tag matching

Some of the source services, notably Eventful and EventBrite, include per-event tags. When one of these tags matches a tag in the (evolving) core vocabulary for that hub, the elmcity service adds that tag to the event’s list of tags which it inherited from its feed.

There are also tables for each foreign service that map tags used there to tags in the hub’s core vocabulary. So, for example, the Eventful tag movies_film and the EventBrite tag movies both map to the core tag film.

As we saw in Portable tags, some iCalendar feeds use the CATEGORIES property of the iCalendar format to express per-event tags. Managing these tags is trickier because, well, they’re unmanaged. Until recently I was suppressing them. Now I’m experimentally allowing them to appear, but segregating them from the core vocabulary. If you check the tags for Boston or Seattle or another city you’ll see that the list divides into two sections. The first presents managed tags: the core vocabulary. The second presents unmanaged tags from iCalendar feeds, enclosed in squiggly brackets to differentiate them from the core vocabulary.

Here’s the current set of unmanaged tags for Boston and Seattle:

When one of these tags matches a tag in a hub’s core vocabulary I promote it — that is, I treat it as part of the managed core and it no longer shows up in squigglies. That’s a top-down approach. But there’s a complementary bottom-up approach. As I scan the unmanaged tags, both within and across hubs, it can become clear that an unmanaged tag belongs in the managed core. To accomplish that I simply use the unmanaged tag somewhere in the managed core. From then on, occurrences of the unmanaged tag are promoted into the core.

A logical next step is to enable curators to edit per-hub maps so that, for example, Seattle’s {central library} and Boston’s {libraries} will be promoted to simply library. I haven’t built this mapping feature yet but it’s on the todo list.

I’m still exploring the interplay between the top-down and bottom-up approaches. But it definitely feels like the right way to handle common vocabularies augmented by different (and regionally-varying) vocabularies.

(This series: elmcity tagging principles.)

Last month I was looking over the shoulder of my auto mechanic, Jonah, when he was retrieving my service record on his computer. I watched him search for udell and find a file called something like 2011-11-04_udell.odf. (He uses an Open Office spreadsheet to keep track of things.) The first thing Jonah did, upon opening the file, was rename it to 2012-01-14_udell.odf. My thought was: I wish we could teach more people how (and why) to do that.

Jonah’s strategy tags each .ODF file with two items of information: a customer name, and a date. His convention is to keep the date current, so that current projects float to the top in date-ordered folder views. For many people the names of files in a folder are just one unorganized namespace. For Jonah they represent two parallel namespaces — or, as I encourage people to think of it, two sets of tags.

One of the benefits of this approach is portability. He could, if needed, transfer those files to another computer, perhaps even one running another operating system, without losing his ability to organize and retrieve records by customer name and date.

Principle: Create and use portable tags

For calendars, the CATEGORIES property of the iCalendar format is the most obvious way to tag events. Unfortunately it isn’t portable. Some content management systems enable users to tag events using the CATEGORIES property. And some calendar applications, like Outlook, also do. But other calendar apps, like Google Calendar and Hotmail Calendar, don’t. If you’re using one of these to publish a calendar, you can’t tag an event as a concert. And if you’re viewing a calendar that has events tagged that way, you won’t see or be able to make use of the concert tag.

There’s a simple and portable solution. iCalendar’s SUMMARY property, which is the title of an event, is universally readable and writable. So if your event stream naturally divides into concerts and lectures, it’s really helpful to identify events accordingly in their titles:

Concert: Joey Pratt Album Release Party with Noah Lefebvre

Lecture: Technology Future Shock: Society, Policy and Innovation in the Digital World

An even better strategy is to provide two separate feeds, one for concerts and the other for lectures. But that’s for a future installment. The key point here is that you can add value to any namespace — a set of files in a folder, a set of events on a calendar — by using tags to qualify filenames or titles.

Mechanism: Use iCalendar filters to extract tag-based feeds

The elmcity service provides a growing set of filters that can extract subsets of iCalendar feeds based on tags found in the SUMMARY (title) or DESCRIPTION (or URL) properties of events.

In the ideal scenario, providers of feeds would use tags as prefixes to the SUMMARY property. In the real world that doesn’t happen, at least not yet. But the elmcity filter is still useful because it’s natural to include keywords in titles and descriptions. Consider, for example, the calendar for Vinology, a wine bar and restaurant in Ann Arbor. Its calendar mixes two different kinds of events. Some are about food and drink (“small plate special”, “happy hour”). Others are about the jazz acts often appearing at Vinology. By filtering on jazz in the SUMMARY and/or DESCRIPTION of Vinology’s Google calendar, the elmcity service is able to extract just the jazz events and add them to Ann Arbor’s music and jazz calendars.

Currently there’s no incentive for Vinology (or anyone else) to adopt this strategy in a more intentional way. That’s because Ann Arbor’s elmcity syndication hub isn’t aligned with attention hubs like AnnArbor.com and ArborWeb.com. If Vinology knew that events tagged with music and/or jazz would show up on those sites in those categories, there would be a strong reason to do it.

(This series: elmcity tagging principles.)

PS: The next Vinology event in the music view of Ann Arbor’s elmcity hub, by the way, is the Doug Horn Trio, this Thursday at 9PM. That event isn’t on the AnnArbor.com calendar or the ArborWeb calendar. To put it there, Vinology would have to take data that it has already entered here and reenter it here and here. I think those other calendars should syndicate the data straight from Vinology (and everyone else).

PPS: See also Harry Tuttle’s busy month and The art of organizing search results.

Back in May I asked: Can elmcity and Delicious continue their partnership? The answer turned out to be no. That’s partly because the new Delicious broke some capabilities I was relying on. But it’s mainly because tagging is so fundamental to the elmcity service that I needed to be able to control, explore, and evolve it.

It continues to evolve, but now’s a good time to review — from the perspective of elmcity curators and contributors — how the principles and mechanisms for tagging calendar feeds (and individual events) illustrate (and extend) some ideas I originally developed during a long infatuation with Delicious. I have a lot to say on this subject, so my plan is to say it in a series of installments of which this is the first.

Principle: Describe things in both general and specific terms

For university calendars, I advise curators to use both a general tag, university, and a specific one. In the case of Seattle some specific tags are uw for the University of Washington, seattleu for Seattle University, and nscc for North Seattle Community College. That makes these views available:

All university-related events, a view that’s currently based on this set of feeds:

Just UW events, based on these feeds:

Just NSCC events, based on these feeds:

Just Seattle U events, based on these feeds:

Mechanism: Multi-tag query

The views above are all based on single-tag queries:

view=university

view=uw

view=nscc

view=seattleu

Here are some examples of multi-tag queries:

view=university,sports (all university sports)

view=seattleu,sports (just Seattle U sports)

view=seattleu,swimming (just Seattle U swimming)

view=university,basketball (all university basketball events)

The last two examples again illustrate the general/specific idea. For sporting events I recommend using the general tag sports and specific tags like swimming and basketball.

Back in 2006, in Del.icio.us is a database, I wrote:

Although it’s intuitively obvious to me, I suspect that most people don’t yet appreciate how easily, and powerfully, tagging systems can work as databases for personal (yet shareable) information management.

Del.icio.us isn’t simply backed by a database, it can function as a database.

I think most people still don’t appreciate that possibility. In the elmcity context I’m hoping to show how it applies not only to personal but also to collective information management.

(This series: elmcity tagging principles.)

In yesterday’s Keene Sentinel guest editorial, YES: Press for more domestic oil, Andrew Morriss consistently misreports oil production figures. Saudi Arabia: 8.1 billion barrels/day; Russia: 10.4 billion; America: 7.9 billion; Iran: 3.7 billion. Those numbers struck me as wrong by orders of magnitude. So I consulted my favorite online scientific fact-checking tool: WolframAlpha.

Q: What is 7.9 billion barrels of oil in dollars? A: $1.05 trillion.

In other words, if we were to convert our daily production to dollars we could pay off the national debt in 15 days. (Real answer: 41 years.)

Q: What is 7.9 billion barrels in gallons? A: 330 billion gallons.

In other words, America’s daily oil production is 4 times the volume of oil transported worldwide in a year. (Real answer: 2.5 times the volume of one of the largest supertankers.)

Q: What is the energy equivalent of 7.9 billion barrels of oil in BTU? A: 45 quadrillion BTU.

In other words, America’s daily oil production is almost half of the total energy consumed by the US in 2001. (Real answer: 1/1000 of that amount.)

What Andrew Morriss meant to write — four times — was million, not billion. Doing that once might be a typo. Doing it four times makes me question his energy literacy.

Now to be fair, most of us — myself included — are not engineers or scientists. We don’t regularly deal with large numbers, and we don’t intuitively grasp relationships among the quantities, prices, and energy content of the various sources that might power our civilization. But we can’t afford to be illiterate (and innumerate) on the topic of energy. The future of the economy, the environment, and of world politics all hinge on our ability to reason about those relationships. Fortunately there is now a tool that can help us all do that more effectively. WolframAlpha isn’t just a boutique search engine. It’s a compendium of scientific knowledge mated to a scientific calculator that understands questions in plain English. And it frames the answers in ways that make sense to everybody. I commend it to Andrew Morriss, to fact-checkers at the Sentinel, and to readers. For more examples of how WolframAlpha can help us reason about energy, see http://delicious.com/judell/energyliteracy.

As we increasingly augment our minds I sometimes pause to reflect on the trade-offs we are making. What powers does the unaugmented mind possess? What do we give up when we outsource our memories to the collective electronic mind? In Dilemma of a Cyborg Carina Chocano writes:

For everything that’s gained by our ability to store and maintain more information than ever before, something is lost that has to do with texture, context and association. The science journalist Joshua Foer, author of “Moonwalking With Einstein: The Art and Science of Remembering Everything,” said in a lecture to the Royal Society of Arts that people once “invested in their memories, they cultivated them. They studiously furnished their minds. They remembered. Today, of course, we’ve got books and computers and smartphones to hold our memories for us. We’ve outsourced our memories to external devices. The result is that we no longer trust our memories. We see every small, forgotten thing as evidence that they’re failing us altogether.” As we store more and more of what makes us us outside of ourselves, he said, “we’ve forgotten how to remember.”

The mnemonic techniques rediscovered in Moonwalking with Einstein were first popularized by Cicero. You bind memories to images, and then you bind the images to a path through the rooms and hallways of a “memory palace.”

Here’s another technique that isn’t so well known. I attribute it to Carlton Fisk by way of a story I heard from the baseball writer Roger Angell. Somewhere in the 2000s, Angell asked Fisk to reflect on what had most altered the game of baseball since his playing days. The salaries? The drugs?

No. The game-changer, Fisk said, was instant replay. His game-winning 1975 home run is one of most-remembered moments in all of sports. The video of that event is one of the most-watched clips. You might think that Carlton Fisk has seen that clip a million times. But in fact, he told Roger Angell, he never watches it. That’s because he doesn’t want to overwrite the original memory, which is his alone, recorded from a point of view that was his alone, with a memory we all share that was recorded by a camera up in the stands.

We can’t do away with instant replay, nor do we want to. But it’s worth remembering how to experience life, even when we know it’s being recorded externally, as if the only cameras are the ones in our heads.

The other day Tim Bray tweeted a Google+ item entitled Geeks and repetitive tasks along with the comment: “Geeks win, eventually.”

Here’s the chart posted on Google+ by Bruno Oliveira:

A couple of things bothered me about this. First, there’s the adversarial tone. The subtext is a favorite geek quotation:

First they ignore you, then they laugh at you, then they fight you, then you win.

That gem is often attributed to Gandhi. Wikiquote disputes that and finds a close variant in Nicholas Klein’s 1918 speech to the Amalgamated Clothing Workers of America. Either way it’s the story of a persistent underdog overcoming an oppressor.

In geek ideology the oppressors are pointy-haired bosses and clueless users. Geeks believe (correctly) that clueless users can’t imagine, never mind implement, automated improvements to repetitive manual chores. The chart divides the world into geeks and non-geeks, and it portrays software-assisted process improvement as a contest that geeks eventually win. This Manichean worldview is unhelpful.

But the chart also fails to capture the reality of repetition and automation in the realm of information systems. Here’s an alternative world view that I choose to imagine and strive to create:

In this view of the world, tasks that involve data manipulation (as so many modern chores do) are undertaken by teams. There is an infinite supply of manual chores. Everybody tackles them. Ideally there is one member of the team I call the toolsmith. Working shoulder to shoulder with the team, the toolsmith spots an opportunity to automate some piece of the work, writes some code, deploys it, observes how it gets used (or doesn’t get used), assesses its impact (or lack of impact), and then iterates on the code. Meanwhile the toolsmith keeps working alongside the team, chipping away at the never-ending and always-evolving list of manual chores, looking for more opportunities to automate, and exploiting them in an incremental and collaborative way.

Software-assisted automation of repetitive work isn’t an event, it’s a process. And if you see it as a contest with winners and losers you are, in my view, doing it wrong.

Before my podcast went AWOL I had been meaning to interview Toby Considine about his efforts to mesh schedules for things with schedules for people, and to define Internet calendaring extensions for that purpose. So when Phil Windley wrote to ask me how I thought calendaring might relate to the personal event networks he wants to bootstrap, I suggested that we ask Toby on an episode of Phil’s Technometria podcast.

Phil, I suspect, is actually one of five identical quintuplets. One teaches at BYU. Another runs a startup called Kynetx. A couple of others take care of family, church, and political matters. Finally there’s the one who does a podcast and also serves as executive producer of ITConversations. I don’t think I could handle being a quintuplet but it’s lucky for all of us that Phil can!

It was a real pleasure to meet Toby and learn about his comprehensive vision for a world in which our buildings, our cars, and our energy grid work with us, appearing as the intelligent agents that science fiction always encouraged us to imagine. One of his favorite examples involves a meeting room that’s scheduled for a 9AM meeting. A really intelligent system wouldn’t start heating or cooling the room at 9AM, it would start sooner. How much sooner? That would depend on the number of people attending.

Orchestrating that kind of dance requires the sort of loosely-coupled event-driven programming at the heart of the Kynetx technology that Phil is creating. Toby is layering that orchestration on top of existing Internet standards: iCalendar (for events), vCard (for resources), LDAP (for directory services). I love that approach for two reasons. First, I’m forever being reminded that we have barely scratched the surface of what might be accomplished with these existing standards if we really put them to use. Second, I violently agree with Toby that automated systems need to build on the standards that people actually use.

Toby chairs the OASIS Web Services Calendar Technical Committee and is an editor of the WS-Calendar specification. If standards whose names begin with WS- give you the heebie-jeebies, take a deep breath. I guess at some point there will be a SOAP profile for WS-Calendar but for now it’s a straightforward set of iCalendar extensions that define intervals, sequences, and relationships.

The web succeeds, in part, because its atomic particle — the hyperlink — can be manipulated both by automated systems and by people. When systems use hyperlinks it’s called RESTful web services. When people do it’s called emailing links. We tend to forget how profoundly the dual nature of the hyperlink binds the web together.

The hyperlink is atomic with respect to location. It says where. The fundamental particle of calendaring — iCalendar’s VEVENT — is atomic with respect to time. It says when. To me it makes perfect sense that smart buildings and energy microgrids will schedule their interactions with us in the same way that we schedule dentist appointments and soccer games.

As I build out the elmcity network, launching calendar hubs in towns and cities around the country, I’ve been gathering examples of excellent web thinking. In Ann Arbor’s public schools are thinking like the web I noted that the schools in that town — and most particularly the Slauson Middle School — are Doing It Right with respect to online calendars. How, I wondered, does that happen? How does a middle school figure out a solution that eludes most universities, theaters, city governments, nightclubs, museums, and other organizations with calendars of interest to the public?

It’s not technology. Slauson Middle School is using the same web services (in this case, Google Calendar) available to everyone.

It’s not budget. The web services required for this solution are free.

It’s a way of thinking. I wrote to Slauson’s principal, Chris Curtis, to congratulate him on the excellent example his school is setting, and to identify the thinker responsible. That thinker turns out to be Chris Curtis himself. And it’s no accident that the implementation pattern on display at Slauson is also evident at Pioneer High. Chris did the same thing there before coming to Slauson.

Now I am not an educator, I only watch from the sidelines. But to me the K-12 “computer skills” curriculum seems uniformed by the kinds of core principles that will make students effective in a web-augmented world. So I asked Chris:

What you’ve done at Pioneer and now Slauson builds on an important conceptual foundation. Do you think that K-12 education could build that foundation?

Here’s what he said:

I agree with the notion that the basic principles of computer science should be generalized more broadly across the curriculum. In many ways, teaching computer and technology skills courses absent a meaningful application of them is ineffective and silly. We wouldn’t teach driver’s education and not let students drive. We don’t teach a “pencil skills class” in which we learn the skills for using this technology tool without an immediate opportunity to apply the skills and then begin to consider and explore the many ways that the pencil and writing change how we organize, perceive, and interact with our world.

This issue gets at the heart of the challenge of technology and education. Often the world seems to divide into separate interest areas: those interested in technology and those interested in education. The result is often to send the technology nerds to a room and make them teach technology and send the other teachers to their rooms and let them teach. In order to be effective at integrating technology into the instructional environment there has to be a merger between a technology interest and and educational interest, within the same person. The awareness of what is possible via technology resources and the desire to perform educational functions can lead to the educator realizing that a task could be done differently, more efficiently, more effectively, with more precision, or in some other manner improved.

Of course the schism that separates technologists from educators also affects practitioners of all kinds. In his most recent essay, Bret Victor meditates on this point:

My piano teacher played the piano. Like, all the time. He had to; it’s not easy to make a living as a musician. Between tours, his band played restaurants, bars, weddings, anywhere they could get a gig. He chose this life because he loved music, and when he taught music, he was teaching what he did. In that way, his teaching was honest.

…

Back in high school, I was taught differential equations by a working engineer. He spent his days at Lawrence Livermore Laboratory, and for whatever reason, chose to spend his evenings at the local community college. Differential equations wasn’t some abstract arcana to him. It was his bread-and-butter, and he apparently found it important enough to share.

…

My information theory professor would teach me information theory in the morning, and then spend the afternoon furthering the field. Sure, what she taught was somewhat elementary by her standards, but she was well aware that this elementary theory was the foundation on which her life’s research was built. It showed, and it stayed with me.

…

Real teaching is not about transferring “the material”, as if knowledge were some sort of mass-produced commodity that ships from Amazon. Real teaching is about conveying a way of thinking. How can a teacher convey a way of thinking when he doesn’t genuinely think that way?

I’m preoccupied with a related question. The way of thinking that I most want to convey is web thinking. Which is, by definition, openly available to anyone who wants to learn. Schools everywhere can observe and emulate what Chris Curtis is doing at Slauson. In so doing they can become practitioners in the way that Chris is. Their students might then see them as practitioners and learn from their examples.

I would be delighted if the elmcity project could help bootstrap that virtuous cycle.

John Ochsendorf, who teaches civil and environmental engineering and architecture at MIT, cares about ancient construction methods and the forgotten knowledge they embodied: rammed-earth walls, braided-fiber suspension bridges, Gothic cathedrals. In an enlightening lunchtime talk a couple of years ago he reviewed what these ancient builders knew and could still teach us if we were willing to learn.

Dave Winer’s Why apps are not the future reminded me of that. The ancient construction method we are forgetting, in this case, is linking. We had already started to forget back in 2004. Then, writing for InfoWorld, I argued that rich Internet apps needed to learn what web apps always knew: RESTful design that exposes internal state as links. Kevin Lynch, who was then chief software architect of Macromedia, accepted the challenge and showed how a Flash app could behave that way.

But while you can bolt RESTful design and linking onto a rich Internet app, or a phone app, these principles aren’t built in. So apps are inherently less open to the sort of lightweight service composition that I’ve championed for a decade, ever since the LibraryLookup project showed me just how easily and powerfully a humble hyperlink can connect two otherwise separate information silos.

The point about service composition and linking is so obvious that we often fail to notice it. Yesterday I booked a hotel in Chicago for an upcoming conference. The organizer sent me an email containing this link for hotel booking:

http://www.marriott.com/hotels/travel/chidm-chicago-marriott/?toDate=1/22/12&groupCode=GROUPCODE&fromDate=1/20/12&app=resvlink

When I followed the link I landed in Marriot’s service with context appropriate to my status as an attendee. Nobody at Marriott specifically designed that capability. It’s just something that’s possible because the site works RESTfully, exposing its state as links that people can hand around.

Now to be honest, you have to be a geek-minded person — like the guy who invited me to this conference — to recognize and exploit this possibility. Most people don’t expect things to work this way, and most web designers don’t try to encourage that expectation.

My fear with rich Internet apps, and now phone apps, is that they tend to obscure an important lesson we never learned well enough in the first place.

It’s been a busy month for rogue plumber Harry Tuttle. When last heard from, he was deploying a filter to fix broken iCalendar feeds served up by the University of Michigan. His next assignment was at Rice University, where the downloadable sports schedules are available only in CSV (comma-separated values) format. Here are the instructions for how to get one of those CSV files into Outloook:

1. Select the File / Import and Export… Menu option.
2. Select “Import from another program or file” and click Next.
3. Select “Comma Separated Values (Windows)” and click Next.
4. Find the desired CSV file and select “Do not import duplicate items” and click Next.
5. Select Calendar and click Next.
6. The next window will have a title that says ‘The following actions will be performed:’ above an option to ‘Import “(filename).csv” into folder: Calendar.’
7. Check that box and and click on ‘Import “(filename).csv” into folder: Calendar’ From there, click and drag needed fields from the left window into the right window. Its fairly self-explanatory, but the basic corresponding fields are below: SUBJECT to Subject START_DATE to Start Date START_TIME to Start Time END_TIME to End Time DESCRIPTION to Description LOCATION to Location

It’s hard to believe any Outlook user has ever followed those instructions. And if you’re using a calendar other than Outlook, you’re on your own.

So I asked Harry to make a filter that turns those CSV files into iCalendar feeds. It’s specific to that particular format for now, but as we find other examples in the wild we’ll generalize accordingly.

My next challenge involved feed categorization. I’m finding pockets of excellence in various places around the country. As I mentioned last month, the Slauson Middle School in Ann Arbor is one of them. Its calendar looks like a single feed, but is actually a merged set of 11 distinct Google Calendars, including one for Slauson clubs, several for Slauson sports, and one for Slauson’s upcoming Chicago trip.

As curator of Ann Arbor’s community hub I was thrilled to see this kind of self-categorization. It enables me to merge Slauson’s sports and music events with other sports and music events in Ann Arbor. But if it’s rare to find a public calendar in machine-readable iCalendar format, which it unfortunately is, it’s even rarer to find a set of machine-readable public calendars that can be syndicated by category.

I asked Harry to weigh in on this problem too. He’s a do-it-yourself guy who doesn’t like to wait for Central Services to take care of things. “If the sources aren’t providing you with the categories you need,” he said, “maybe we can create them ourselves.” Next thing you know, he’d whipped up a filter that selects subsets of iCalendar feeds by keyword and/or time.

Here’s one example of how I’m using it. Vinology, a restaurant and wine bar in Ann Arbor, publishes upcoming events using a Google Calendar. The feed from that calendar mixes two different kinds of events: restaurant business (Happy Hour) and a jazz schedule (Meri Slaven trio). I’d like to select just the jazz events for Ann Arbor’s music and jazz views.

Harry’s filter solves this problem. Actually, it solves it in two different ways. The first way is to filter Vinology’s feed for events where the SUMMARY or DESCRIPTION fields contain the keyword jazz. The second way is to filter the feed for events that happen after 8PM. Either of these methods produces the jazz-categorized iCalendar feed that Vinology might someday create for itself.

Harry’s filter also solves a related problem. The Berkeley Art Museum / Pacific Film Archive is, as the name suggests, a venue for both art exhibits and film showings. The calendar is, happily, available as an iCalendar feed. Unhappily it mixes art exhibits with film showings and I couldn’t find any times or keywords that would reliably separate the two streams. But then I found a keyword that would. The BAM/PFA website’s URLs look like http://www.bampfa.berkeley.edu/exhibition/faulders_bamscape) and http://www.bampfa.berkeley.edu/filmseries/clouzot). And those same URLs appear in the iCalendar feed. So Harry enhanced the filter to look for those boldedkeywords in the URL field of iCalendar events. Now the BAM/PFA’s film series shows up where it belongs, in the film category of the Berkeley hub.

Senator Ted Stevens got it right. The Internet really is a series of tubes. When they don’t fit together quite the way you need them to, Harry Tuttle can sometimes help.

As we saw last week, the future of community-scale calendaring is already here in some cases but isn’t yet evenly distributed. Consider the Seattle Aquarium’s public calendar. As curator of Seattle’s elmcity hub I’ve found four iCalendar feeds for the Aquarium:

1. An Eventful venue. (iCal feed.)

2. An Upcoming venue. (iCal feed.)

3. A Facebook Page. (iCal feed.)

4. The Aquarium’s own calendar page. (iCal feed.)

If you visit #4, the Aquarium’s calendar page, you’ll see that it does offer an Export as iCalendar link. So why isn’t that link hot here? Because it’s only wired to JavaScript code that imports a snapshot of the data into a calendar program. No URL points directly to the data. That means people can’t subscribe to the feed — and neither can the Seattle hub.

If the Aquarium were truly thinking like the web it would offer its calendar as a first-class web resource addressed by an iCalendar URL. Even better, it would offer one such URL for each view of the calendar page: Educators, Family / Kids, Members Only, Out and About, Speaker Series, Special Events, Summer Camps, Volunteer Orientation.

Since those views aren’t available as feeds I had to fall back to the other three sources. At first I merged all three to create the aquarium view of the Seattle hub. When I compared the results of the merge, though, something didn’t cross-check. An event called Marine Mammal Talk, which showed up in the Upcoming feed, wasn’t in the Eventful or Facebook feeds or on the Aquarium’s own page. Yet Upcoming’s Marine Mammal Talk page says this event was discovered by the Upcoming Robot, has occurred 498 times already, and will occur 77 more times. What’s up with that?

I searched the Aquarium’s site and found the event. Here are the details:

Tue, 15 Nov, 2011 2:43 PM – 2:43 PM

Three weekends of special activities all about the Aquarium’s Marine Mammals – Otters, Harbor Seals & Fur Seals. Join us for special talks, demonstrations and hands-on activities.

Fees: Included with regular Aquarium Admission

Calendar: Programs & Events Calendar

Category: Special Events

Repeats: Weekly on Sunday, Friday, Saturday until 4/24/2011

You can see why the Upcoming Robot was confused. The event’s date is given as Tue 15 Nov 2011. But its recurrence rule says that the last recurrence was back in April of this year.

Of course this fragment of text doesn’t really specify a date or a recurrence rule. The Upcoming Robot just inferred those, erroneously, from unstructured text. And then, like the Sorcerer’s Apprentice, it began pumping those errors onto the web. It has evidently done so for a couple of years and will, if unchecked, continue into 2013.

I don’t blame the Upcoming Robot. Denied access to the structured data it craved, the poor thing scavenged what it could. The lesson here is for the Seattle Aquarium and for all who mean to publish data online. If you don’t establish yourself as the authoritative source for that data then others will step in to do it for you. And they are liable to get it wrong.

As I review and improve the elmcity hubs in selected cities, I am again reminded of William Gibson’s wonderful aphorism: “The future is already here, it’s just not evenly distributed.” Yesterday we saw that the future of community calendars hasn’t yet arrived at the University of Michigan. But today I was delighted to see that it has arrived, in a big way, for the Ann Arbor public schools. Almost all of them, it turns out, are making good use of Google Calendar to publish machine-readable calendar information. This morning I rounded up thirty of those calendars and added them to Ann Arbor’s elmcity hub, bringing the total number of feeds from 194 to 224.

Here’s the breakdown of the 309 events from the grade schools:

And the 322 events from the middle schools:

And the 966 events from the high schools:

Among grade schools, Slauson is notable not only for the number of events but for the exemplary self-categorization applied to them. When you click the Google Calendar subscribe button on the Slauson calendar page here’s what you’ll see:

This is a best practice I wish everyone would adopt. It illustrates the seventh of my seven ways to think like the web: #7: Reuse components and services. The Slauson calendar is both a user of other services (the district-wide calendars) and a provider of services. And as a provider, it understands the idea of componentization. In an era of abundance it costs no more to create and manage a dozen calendars, using free services like Google Calendar and Hotmail Calendar, than to jam everything into a single calendar. The benefits are manifold. They include:

Delegation

In most schools and businesses, maintenance of “the” public calendar is one person’s job. That person becomes a bottleneck. When you recognize that logically there isn’t one public calendar, but instead several or many, each with its own appropriate maintainer, then you can break that bottleneck.

Precision

A parent who subscribes to a single undifferentiated school calendar may be overwhelmed by the flow. Parents of kids who are in music programs, or on sports teams, or who are going on the Chicago trip, should be able to focus on those activities.

Scope

Slauson Middle School is part of the larger Ann Arbor community. When Slauson’s calendars are self-categorized, they can align with community-wide views. Here’s a picture of some sports-related activites in Ann Arbor on November 17th:

Slauson’s publication of a set of self-categorized machine-readable calendar feeds enables it to represent its sports activities on a city-wide timeline that includes, in this particular view, events from the Ann Arbor Triathlon Club, the Wolverines basketball teams, and Ann Arbor’s kickball Meetup group.

Well done Slauson! And kudos to all the Ann Arbor public schools. You have become web thinkers. I hope schools everywhere will learn from your example.

Here’s one of my favorite scenes from the movie Brazil,

Sam Lowry (Jonathan Pryce): Are you from central services?

Harry Tuttle (Robert De Niro): Hah! They’re a little overworked these days. Luckily I intercepted your call.

Sam Lowry: Can you fix it?

Harry Tuttle: No, but I can bypass it with one of these.

Back in 2003, my InfoWorld column APIs, protocols, and rogue plumbers made three points:

1. The web of data shouldn’t require the services of Harry Tuttle.

2. Unfortunately it still does.

3. Fortunately the web’s architecture (a series of tubes) enables Harry to intervene when he must.

Those points are all still valid in 2011. From time to time I still have Harry Tuttle moments. Today’s involved the campus events system at my alma mater, the University of Michigan. In the current phase of the elmcity project I’m rebuilding hubs in various cities in order to dramatically beef up the number of feeds and quality of tagging. Ann Arbor’s hub was conspicuously lacking feeds from the University of Michigan. When I investigated I found that the central service, http://events.umich.edu, does offer iCalendar feeds. Yay! You can’t take that for granted, many if not most schools don’t make their public calendars available in a machine-readable way. Unfortunately there’s a problem with the feeds produced by the UM service. They’re invalid. You can’t load them into an iCalendar-aware program like Google Calendar or Outlook, and the elmcity engine can’t aggregate them.

I reported the problem to central services and have been awaiting a fix for some time. Today, because I wanted to get my hands on that data, I unleashed Harry Tuttle. There are two major problems with the iCalendar export from events.umich.edu. First, the lines of text aren’t properly folded. Second, the timezone properties don’t refer to a timezone definition. So I made a filter that fixes these major problems (plus some other minor ones). Here’s what the filter does:

Original: http://jonudell.net/data/failed-ics-umich.ics.txt. Validation result: Unparseable.

Line-folding fixed: http://jonudell.net/data/fixed-ics-umich.ics.txt. Validation result: Better.

VTIMEZONE added: http://jonudell.net/data/clean-ics-umich.ics.txt. Validation result: Valid.

Then I used the filter to add a bunch of feeds to the hub. Here’s one for the Taubman Health Services Libraries, and another for the Gerald R. Ford Presidential Library. These merge with other library feeds, notably from the Ann Arbor District Library, in the hub’s top-level library view.

Now, where do I fill out that twenty-seven-B-stroke-six form?

From Wikipedia’s Brazil_(film) page:

The reference to form 27B-6, without which no work can be done by repairmen of the Department of Public Works, is a reference to George Orwell, who lived at 27B Canonbury Square Apartment 6, while writing Nineteen Eighty-Four.

In a pair of recent entries, Semantic web 101: Say what you mean and The long tail of the iCalendar ecosystem, I’ve begun to report on what I’m learning about the state of the iCalendar ecosystem as I work in parallel on the elmcity project and on the iCalendar Validator. Today I’ll focus on just one of a number of issues I’ve run into. Consider these two screenshots:

google calendar	hotmail calendar

On the left you see Google Calendar displaying two calendars, each representing a single event — the Brower Youth Awards on October 18 at the Herbst Theater in San Francisco — in a different way. On the right you see Hotmail Calendar displaying the same two calendars. The event will happen at 5:30 Pacific time on the 18th. I found it on the Berkeleyside site whose events page offers a companion iCalendar feed.

If you load that feed into both Google Calendar and Hotmail Calendar, and if your calendars are set to Eastern time, you’ll see what’s shown above. If your calendars are set to another timezone the times will be shifted but the pink ones still won’t match.

The green ones should always match and should always be what you’d expect. For me, looking at a 5:30 Pacific event through the lense of calendars set to Eastern, I’d expect both calendars to display 8:30.

What’s the difference between the pink calendar and the green calendar? Here’s the pink one. It’s just the original calendar reduced to a single event. Like the original it declares its timezone using the nonstandard X-WR-TIMEZONE property:

BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
CALSCALE:GREGORIAN
PRODID:-//Refresh Web Development//Helios Calendar//EN
X-FROM-URL:http://www.berkeleyside.com/BerkeleysideCalendar/events/
X-WR-RELCALID:Berkeleyside
X-WR-CALNAME:Berkeleyside
X-WR-TIMEZONE:America/Los_Angeles
BEGIN:VEVENT
URL;VALUE=URI:http://www.berkeleyside.com/BerkeleysideCalendar/events/index.php?com=detail&eID=6
DTSTART:20111018T173000
DTEND:20111018T210000
SUMMARY:Brower Youth Awards 2011
LOCATION:Herbst Theater - 401 Van Ness \, San Francisco\, CA US 94102
END:VEVENT
END:VCALENDAR

And here’s the green one. Again it’s a derivation of the original calendar that reduces to a single event. But it also declares its timezone in the standard way, using a VTIMEZONE component and then referring to it using the TZID parameter of the DTSTART and DTEND properties:

BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
CALSCALE:GREGORIAN
PRODID:-//Refresh Web Development//Helios Calendar//EN
X-FROM-URL:http://www.berkeleyside.com/BerkeleysideCalendar/events/
X-WR-RELCALID:Berkeleyside
X-WR-CALNAME:Berkeleyside
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
URL;VALUE=URI:http://www.berkeleyside.com/BerkeleysideCalendar/events/index.php?com=detail&eID=6
DTSTART;TZID=America/Los_Angeles:20111018T173000
DTEND;TZID=America/Los_Angeles:20111018T210000
SUMMARY:Brower Youth Awards 2011
LOCATION:Herbst Theater - 401 Van Ness \, San Francisco\, CA US 94102
END:VEVENT
END:VCALENDAR

As we see in the picture above, the event time on the green calendar shows up the same way in both Google Calendar and Hotmail Calendar. It should also be the same in any calendar program that supports the iCalendar standard.

The event time on the pink calendar, though, is a up for grabs. Calendar programs that strictly follow the iCalendar standard should ignore X-WR-TIMEZONE and always display the local time, 5:30PM, which will be right for people in the Pacific timezone and wrong for everybody else. Hotmail Calendar does this. Programs that use X-WR-TIMEZONE, on the other hand, can render this calendar just as they would render a standard calendar. Google Calendar does this.

Why do I care? I have to decide whether the elmcity service will or won’t consider X-WR-TIMEZONE to be meaningful. The service is based on DDay.iCal, the same standards-based parser that powers the iCalendar Validator. So when the the service reads the pink calendar, and renders it for users in Berkeley, it will do the wrong thing from their point of view.

To do the right thing for Berkeley it would need to do the wrong thing by iCalendar: transform the nonstandard X-WR-TIMEZONE property into a standard VTIMEZONE component, and then transform all the dates so that they refer to the VTIMEZONE’s TZID. In order to create that VTIMEZONE component, it would interpret X-WR-TIMEZONE value as a TZID (timezone ID) from the Olson database. A Unix-based service would look up the TZID in Olson, find the rule for the timezone — i.e., offsets from GMT for standard time and daylight savings time, and when to appy them — and express the rule using VTIMEZONE syntax. A service running on Windows Azure, like mine, would instead need to map the Olson name to a Windows timezone name, look up the rule using a Windows API, and then express the rule in VTIMEZONE syntax.

Of course this is a slippery slope because, in the end, I’m only guessing what X-WR-TIMEZONE is supposed to mean. Here’s Rick DeNatale engaging in the same kind of guesswork:

Someone pointed me to this icalendar file of Australian holidays for a test case:

http://icalx.com/public/rohanl/Australian32Holidays.ics

This contains NO VTIMEZONE components, but does have the calendar property: X-WR-TIMEZONE:Australia/Sydney

Googling indicates that this is a non-standardized property, but it seems to be used by several calendar apps including Apple’s ical.app
and Google calendar.

I know that it’s non-standard, but it seems to be somewhat important for interoperability. I’m looking for some kind of information about
what it means in general.

It seems to indicate a default tzid for the whole calendar. In the absence of timezone components I’m not sure how to interpret the tzid, though.

Australia/Sydney IS a time zone identifier in the Olsen database, is it standard practice to use olsen tzids in X-WR-TIMEZONE calendar attributes?

Fortunately I can bring some data to bear on this question. Thanks to the iCalendar Validator I can analyze public calendars produced by a variety of iCalendar producers. In The long tail of the iCalendar ecosystem I listed the names of about 600 producers seen recently by the Validator. Of those, about 100 use X-WR-TIMEZONE instead of VTIMEZONE, and 70 of those 100 use local rather that UTC date syntax which implies they are depending on X-WR-TIMEZONE for correct interpretation of those dates.

Note that Google Calendar, the 800-pound gorilla in this space, is not one of those 70 producers. When it writes iCalendar format it uses both X-WR-TIMEZONE and VTIMEZONE; the latter ensures that Google Calendars can be understood properly by standard parsers that don’t support X-WR-TIMEZONE. The 100 producers I’m talking about, though, are using only X-WR-CALENDAR in a way that suggests they expect a nonstandard transformation. The fact that Google Calendar performs that transformation is, of course, a major reason why producers would expect it to happen everywhere.

Should X-WR-TIMEZONE be standard? That’s debatable. It would certainly make life easier for iCalendar producers. They could just mention a timezone rather than having to extract its rule from their operating systems and express the rule in VTIMEZONE syntax. One of the reasons for the success of RSS and Atom, after all, is that it’s always been easy to whip up an RSS or Atom feed which you can then check with the Feed Validator. An analogous simplicity for iCalendar producers would help grow the iCalendar ecosystem.

On the other hand if an iCalendar feed were to only mention a timezone without fully defining it, then the consumer would have to do the work that the producer didn’t. That’s problematic as Doug Day, author of the iCalendar Validator, notes in a recent email exchange:

Unfortunately, without including the actual time zone information in the calendar (i.e. via VTIMEZONE), you can’t be sure that the date/times you’re representing are accurate, even when using X-WR-TIMEZONE. For example, if you’re on a Windows XP machine that hasn’t been updated in 5 or 6 years, your system time zone information will be inaccurate. However, if the VTIMEZONE were included in the calendar, it would remain accurate, even on an older machine with out-of-date time zone definitions. Also, in order to interpret X-WR-TIMEZONE, you’d need to be in an environment where interpreting Olson time zone is realistic (easy on Linux, harder on Windows). I know a global, online time zone registry is in the works, but I don’t think it’s to a point where it’s useful, yet.

You might wonder why all this timezone stuff is even necessary. After all, an iCalendar feed can simply omit VTIMEZONE (and/or X-WR-TIMEZONE), express dates and times in UTC (Coordinated Universal Time), and use UTC syntax for all dates and times. Why not just do that? I asked Doug Day about this a while ago, and here was his reply:

The biggest problem is with recurring events and daylight/standard time transitions. For example, consider the following (all hypothetical):

1. I live in Salt Lake City, Utah

2. I want to schedule a meeting, starting on September 7, 2009 at 9:00 A.M, which recurs every month on the first Monday.

3. Some of the people attending this meeting live outside my current time zone.

So, here are the occurrences you’re ultimately after:

- September 7, 2009 – 9:00 A.M. (3:00 PM UTC)

- October 5, 2009 – 9:00 A.M. (3:00 PM UTC)

- November 2, 2009 – 9:00 A.M. (4:00 PM UTC)

As you can see, once the time changes from daylight back to standard time, so does the UTC representation of that time. So, if you had scheduled your event in UTC time, when the time zone changes, your event time will actually have changed (to 10:00 A.M., rather than 9:00 A.M.)

For this reason, among others, it’s always best to include time zone information whenever available. Traditionally, it’s been pretty difficult to include that information, and it’s more often left out than included.

So what shall I do with X-WR-TIMEZONE? I’ve decided to support it experimentally. If you start a new hub on the elmcity service, the default is to ignore X-WR-TIMEZONE. But if your hub has important sources that depend on it, as Berkeley does, then you can override the default so the times will be as you expect.

Meanwhile we’re going to update the iCalendar Validator to warn producers about this issue. There’s nothing technically invalid about a calendar that uses X-WR-TIMEZONE without VTIMEZONE. To a parser that strictly interprets the RFC5545 standard, that property is just a name that’s “reserved for experimental use.” But as has always been true of the RSS/ATOM Validator, the iCalendar Validator aims to deliver useful real-world guidance. Producers that use X-WR-TIMEZONE alone to declare a timezone should know that while it may often yield expected results, it’s not guaranteed to do so. It would be better to use a standard VTIMEZONE.

A couple of months ago I began saving the iCalendar files that are submitted to the iCalendar Validator. Today I extracted a list of unique names of iCalendar producers along with associated counts of the number of calendars validated for each. Here they are, with Google Calendar at the head and a classic long tail distribution of almost 600 other iCalendar producers. (You can also see them elsewhere by name as well as by count.)

The next step will be to analyze how well these these producers conform to the validator’s interpretation of the iCalendar spec. But the list itself forms an interesting data set. We know intuitively that, after 12 years of evolution, the iCalendar ecosystem must have become broad and diverse. Here’s a nice illustration of that breadth and diversity.

(Note that these counts don’t necessarily reflect the real distribution of iCalendar producers. The iCalendar Validator is closely associated with the elmcity project, so certain producers used heavily there — DDay.iCal, EVDB, Meetup — are overrepresented. On the whole, though, I’d guess this is a reasonable proxy for the distribution of producers.)

If you think that the semantic web is just some kind of geek rapture, like the singularity, I can understand why. As with Zeno’s Paradox we’re always advancing but never arriving. Unlike the singularity, though, I do expect a semantic web to emerge in my lifetime. The latest initiative is schema.org, sponsored by Google, by Yahoo!, and by my employer, Microsoft. Schema.org recapitulates prior efforts to define how webmasters can mark up web pages to include structured data. I hope that this time, thanks to collaboration among the major search engines, we’ll finally cross the activation threshold.

Meanwhile I’ve been toiling in another part of the semantic web. I’ve been trying to get webmasters people to understand why and how to publish calendars as machine-readable structured data in addition to human-readable text. If you’ve followed the elmcity saga you know I’m on a crusade to make more and better use of the Internet’s venerable standard for exchanging structured calendar events: iCalendar.

It’s been a struggle. Almost every website run by a school, club, business, or town has an Events page. Those pages are, almost always, data siloes: HTML or PDF files that can be read by people but cannot be processed by machines. Only rarely do such pages offer links to iCalendar feeds served up by Google Calendar or Drupal or Hotmail Calendar or some other service capable of producing such feeds. So when elmcity curators discover one of these rare feeds, it’s cause for rejoicing.

Sadly that joy is sometimes short-lived. A surprising number of iCalendar feeds just plain don’t work. That’s why I invited Doug Day to create the iCalendar Validator, a service that helps producers of iCalendar feeds conform to the specification. It’s always painful when I have to explain to a curator that the shiny new feed they’ve discovered doesn’t conform and won’t deliver events to the hub.

Here are three sources of iCalendar feeds that, I’ve recently discovered, don’t work.

1. The University of Michigan’s UM Events. It’s a major hub that serves many campus websites. As far as I can tell, all of those sites are providing feeds with malformed descriptions. Here’s an example of the problem and the fix. (iCalendar producer ID: UM//UM*Events)

2. CiviCRM is a “free, libre and open source software constituent relationship management solution.” Its feeds also have malformed descriptions. Here’s an example of the problem and the fix. (iCalendar producer ID: CiviCRM//NONSGML CiviEvent iCal)

3. Drupal is a popular open source content management system. I’ve seen Drupal feeds used successfully, but today I found one that fails for two reasons: a malformed recurrence rule, and a missing timezone definition. Here’s an example of the problem and the fix. (iCalendar producer ID: Drupal iCal API)

These problems are minor and would be easy to resolve. I’ll try to contact the authors of these iCalendar producers; if you can help put me in touch I’d appreciate that. I’m also going to look through the logs written by the iCalendar Validator, compile a list of producers of invalid feeds, and try to contact them as well.

Where’s the connection to the semantic web? At the end of the day, as RSS/Atom validator co-creator Sam Ruby likes to say, “It’s just data.” But structured data, whether it conforms to the dozen-year-old iCalendar standard or some newfangled microdata standard, is easily screwed up. And the consequences of screwups are often silent. Services that were looking for that structured data find nothing, mutter to themselves, and move along.

As we collectively create the semantic web we’ll need to make sure that the structured data we intend to publish really says what we mean.