Jon Udell

A couple of months ago I began saving the iCalendar files that are submitted to the iCalendar Validator. Today I extracted a list of unique names of iCalendar producers along with associated counts of the number of calendars validated for each. Here they are, with Google Calendar at the head and a classic long tail distribution of almost 600 other iCalendar producers. (You can also see them elsewhere by name as well as by count.)

The next step will be to analyze how well these these producers conform to the validator’s interpretation of the iCalendar spec. But the list itself forms an interesting data set. We know intuitively that, after 12 years of evolution, the iCalendar ecosystem must have become broad and diverse. Here’s a nice illustration of that breadth and diversity.

(Note that these counts don’t necessarily reflect the real distribution of iCalendar producers. The iCalendar Validator is closely associated with the elmcity project, so certain producers used heavily there — DDay.iCal, EVDB, Meetup — are overrepresented. On the whole, though, I’d guess this is a reasonable proxy for the distribution of producers.)

If you think that the semantic web is just some kind of geek rapture, like the singularity, I can understand why. As with Zeno’s Paradox we’re always advancing but never arriving. Unlike the singularity, though, I do expect a semantic web to emerge in my lifetime. The latest initiative is schema.org, sponsored by Google, by Yahoo!, and by my employer, Microsoft. Schema.org recapitulates prior efforts to define how webmasters can mark up web pages to include structured data. I hope that this time, thanks to collaboration among the major search engines, we’ll finally cross the activation threshold.

Meanwhile I’ve been toiling in another part of the semantic web. I’ve been trying to get webmasters people to understand why and how to publish calendars as machine-readable structured data in addition to human-readable text. If you’ve followed the elmcity saga you know I’m on a crusade to make more and better use of the Internet’s venerable standard for exchanging structured calendar events: iCalendar.

It’s been a struggle. Almost every website run by a school, club, business, or town has an Events page. Those pages are, almost always, data siloes: HTML or PDF files that can be read by people but cannot be processed by machines. Only rarely do such pages offer links to iCalendar feeds served up by Google Calendar or Drupal or Hotmail Calendar or some other service capable of producing such feeds. So when elmcity curators discover one of these rare feeds, it’s cause for rejoicing.

Sadly that joy is sometimes short-lived. A surprising number of iCalendar feeds just plain don’t work. That’s why I invited Doug Day to create the iCalendar Validator, a service that helps producers of iCalendar feeds conform to the specification. It’s always painful when I have to explain to a curator that the shiny new feed they’ve discovered doesn’t conform and won’t deliver events to the hub.

Here are three sources of iCalendar feeds that, I’ve recently discovered, don’t work.

1. The University of Michigan’s UM Events. It’s a major hub that serves many campus websites. As far as I can tell, all of those sites are providing feeds with malformed descriptions. Here’s an example of the problem and the fix. (iCalendar producer ID: UM//UM*Events)

2. CiviCRM is a “free, libre and open source software constituent relationship management solution.” Its feeds also have malformed descriptions. Here’s an example of the problem and the fix. (iCalendar producer ID: CiviCRM//NONSGML CiviEvent iCal)

3. Drupal is a popular open source content management system. I’ve seen Drupal feeds used successfully, but today I found one that fails for two reasons: a malformed recurrence rule, and a missing timezone definition. Here’s an example of the problem and the fix. (iCalendar producer ID: Drupal iCal API)

These problems are minor and would be easy to resolve. I’ll try to contact the authors of these iCalendar producers; if you can help put me in touch I’d appreciate that. I’m also going to look through the logs written by the iCalendar Validator, compile a list of producers of invalid feeds, and try to contact them as well.

Where’s the connection to the semantic web? At the end of the day, as RSS/Atom validator co-creator Sam Ruby likes to say, “It’s just data.” But structured data, whether it conforms to the dozen-year-old iCalendar standard or some newfangled microdata standard, is easily screwed up. And the consequences of screwups are often silent. Services that were looking for that structured data find nothing, mutter to themselves, and move along.

As we collectively create the semantic web we’ll need to make sure that the structured data we intend to publish really says what we mean.

If you follow this sort of thing, you already know about IFTTT. It’s a new web service that enables non-programmers to compose other web services. The acronym expands to If This Then That, and here are some ways you can use the metaphor:

If I am tagged in a Facebook photo, then save the photo to Evernote

If the library sends email saying my book on hold is ready, send me a text message

My colleague Scott Hanselman says this is bloody brilliant and I agree. It’s the next step in a journey that began for me back in 1999 when I mashed up Alta Vista’s search engine with Yahoo! directories to measure the mindshare of sites by category. More recently Yahoo! Pipes made service mashing easier for non-programmers. Now IFTTT enables everyone to play. Wonderful!

So why am I less enthusiastic about IFTT than I thought I would be? There are two related reasons. First, here’s what IFTTT says when you ask it to activate its Twitter channel on your behalf:

This application will be able to:

• Read Tweets from your timeline.
• See who you follow, and follow new people.
• Update your profile.
• Post Tweets for you.
• Access your direct messages.

Excellent! This is an example of OAuth, a protocol that enables you to delegate powers to IFTTT without giving up credentials. Most of the services you can use with IFTTT support OAuth, and that represents another huge step forward for the web.

What if I only want to give IFTTT the power to tweet on my behalf, though, and not give up access to my private direct messages? More generally, how can I think about the tradeoffs involved in delegating all versus some versus no powers to IFTTT, across a range of services I might authorize it to use on my behalf?

This leads to the second and broader concern. If I’m not paying for the product, I am the product. As is true for many free services on the web today, I have no contractual relationship with IFTTT. I pay for the service it provides by surrendering access to my data. OAuth helps me negotiate how much access, but if I give up none then IFTTT is powerless.

Why, though, can’t I pay for the product instead of being the product? I want IFTTT to work for me, I want to pay for the service, and in return I want it to promise never to keep or use any of the data it exchanges on my behalf. I realize this may not be a popular option anytime soon. But it’s time to start the conversation. Services don’t only want to be free, they also want to be valuable. That’s rarely a choice nowadays. It needs to become one.

Yesterday’s stream of notifications brought two links paired with invitations for me to comment. The first link points to a NY Times story about how AVOS, the new owner of Delicious, plans to remake that service. Chad Hurley:

The home page would feature browseable “stacks,” or collections of related images, videos and links shared around topical events. The site would also make personalized recommendations for users, based on their sharing habits. “We want to simplify things visually, mainstream the product and make it easier for people to understand what they’re doing,” Mr. Hurley said.

The second link points to a blog post from Mark Surman, executive director of the Mozilla Foundation, who wants to teach the world to code:

This has been the premise behind much of what we have done with Mozilla Drumbeat: people who make stuff on the internet are better creators and better online citizens if they know at least a little bit about the web’s basic building blocks. Even if they only learn a little HTML, the web gets better.

I wish I could broker a conversation between Chad Hurley and Mark Surman. If mainstream folk used Delicious and understood what they’re doing when using it, they’d understand themselves to be makers of things on the Internet. The things they make are custom information systems. They make them by writing code, but not in the languages of HTML or JavaScript. Instead they use tag vocabularies that produce and consume web services. And services, I argue, are the most fundamental of the web’s basic building blocks.

Here is an example: http://www.delicious.com/judell/del.icio.us. Ostensibly it’s a list of dozens of articles I’ve written over the years about what I mean when I say that Delicious enables non-programmers to code and use web services. But it’s not just a list. I think of it as a web service. One aspect of the service provides the list in HTML format for people to read in browsers. Another provides the list in RSS format that enables cooperating services to watch the list and react when new items are added. Another enables the list to combine with other lists. Here, for example, is a subset of my Delicious-related articles that are also related to the elmcity project: http://www.delicious.com/judell/del.icio.us+elmcity.

More than anything before or since, Delicious empowers me to manage web resources — both personally and socially. Once those resources were mainly things we found on the web. Now they’re also things we make on the web. I hope the forthcoming Delicious makeover will help people understand it to be a tool for creating, mixing, and sharing web resources. And I hope it remains the sort of open web tool that Mozilla Drumbeat wants to popularize.

Last September we bought a new dishwasher to replace the old one that had failed. It was a reluctant purchase. We’d actually gone a couple of years doing dishes by hand, partly because we’ve been so disappointed by the modern generation of appliances. When the salesman at Sears mentioned their free 5-year preventive maintenance program, though, we decided to opt in. The plan entitles you to an annual appointment with a dishwasher tech who will come by to inspect your machine and do what’s needed to keep it in good working order.

Stoves, dishwashers, and vacuum cleaners shouldn’t require this kind of life support. I’m not going to win that argument, but I can at least make effective use of the life support service. Doing so requires a bit of a hack, though. Why? Remembering something annually, on a certain date, for five years running, isn’t the sort of thing that humans do well. It’s a task that begs for automation.

Of course Sears could remind me, in September of 2011, 2012, 2013, 2014, and 2015, about my dishwasher checkup. It would be easy for them to do. They’re a mega-corporation with a mighty IT department that can easily perform this feat of magic. But Sears doesn’t want to remind me. They’re betting that I’ll forget, as I’m sure most people do.

So they sounded quite surprised when I called yesterday, on the anniversary of my dishwasher purchase, to schedule my appointment. How did I do it? By casting this magic spell:

BEGIN:VEVENT
DTSTART;VALUE=DATE:20110902
DTEND;VALUE=DATE:20110903
RRULE:FREQ=YEARLY;COUNT=5
SUMMARY:sears dishwasher
END:VEVENT

That’s an event recorded by my calendar program. It happened first on September 2, 2011, and will happen again yearly for five years. I cast my spell using Outlook but you can do the same thing using any calendar program: Google Calendar, Apple iCal, Hotmail Calendar, Lotus Notes, many others.

When I talk about iCalendar feeds in the context of the elmcity project, I tend to focus on the idea that these are data feeds, which they are. But iCalendar has a special property. Some of that data is actually code. In this example, here is the one line of code:

RRULE:FREQ=YEARLY;COUNT=5

In iCalendar lingo, RRULE means recurrence rule. From Outlook’s or Google Calendar’s point of view the rule is a tiny program and they are the operating systems that run it. Here are some other RRULES that calendar applications enable you to express:

every Tuesday at 10AM

every first Sunday until October 2011

every September 2 for five years

You don’t have to do it this way. You can, instead, record each event separately. Even though calendar software makes it quite easy to define repeating events, I’ve got a hunch that in the dishwasher scenario a lot of people would use data (five discrete events) rather than code (one event that expands to five). Why? For the same reason that many people will manually adjust the font size used for every paragraph in a document, rather than creating a rule that governs all the paragraphs.

My list of core principles for web thinkers doesn’t yet include the idea of code that generates data. I’ve found it plenty challenging just to get people to think about the nature of their data, about how they do or don’t control it, and about how it does or doesn’t flow through networks.

But if we want everyone to be able to automate tasks that otherwise get done poorly or not at all, we probably do need to find a way to teach patterns like code-generated data and Don’t Repeat Yourself more broadly than just to students of computer and information sciences.

If you want to read a lovely short essay on calendar software and our relationship to the future, I highly recommend Paul Ford’s Tickler File Forever.