Pub/sub, tags, and human filters

In 2002, InfoWorld gave a Technology of the Year award to “publish/subscribe” technology. In the writeup I mentioned Kenamea, KnowNow, and the Flash Communications Server. The del.icio.us bookmarking system has some of the pub/sub flavor of those systems, as well as some of the blogging flavor.

In the blog network, you publish to a personal identity (your own), and you subscribe to other people’s identities. In systems like KnowNow and Kenamea, people (and also applications) publish to, and subscribe to, topics.

Consider the del.icio.us tag e4x, which I created today to help me keep track of this article on a subject I expect to learn more about soon. At the moment, my e4x page and the systemwide e4x page are the same: mine is the one and only use of that tag.

Even if I’m the only one to collect e4x references by means of that tag, it will have value. I’ll be able to access a set of bookmarks from anywhere, and easily share them. Things could get more interesting if other people’s e4x references start to show up when I visit (or subscribe to) the tag. Whether del.icio.us (or an analogous service) will reach a scale that makes that likely, for specialized as well as common terms, is an interesting question.

Once a tag does reach critical mass, another interesting question arises. Do you monitor the global view or do you rely on one or more user-filtered views? I guess the answer is both, at different times. When a tag is new and receives little traffic, watch the whole thing. If traffic grows too heavy or too noisy, interpose trusted human filters.

Looking back I can see what attracted me to Delicious. It embodies what I’ve come to know as ways of thinking like the web.

I am now working on a service that invites people to learn and apply web thinking in order to systematically inform one another about things. My web service uses Delicious as a partner service. One reason is that I am virtuously lazy. I would much rather use a service than create one. The elmcity project only cares about one thing: calendar syndication. If it can partner with a service like Delicious for other things — managing lists of feeds, configuring hubs — then I can focus on trying to do the one thing that really matters to my project.

But there’s another reason. I believe that people who use Delicious in the way that elmcity curators do are learning to apply some key principles of web thinking. Things like informal contracts, information chemistry, the pub/sub communication pattern, and the structure of information,

I wrote up specific examples recently in Can elmcity and Delicious continue their partnership? Today I realized that I still lack an answer to that question. If the new terms of service are going to require me to swap out Delicious for another service I should get cracking. But first I’ll try again. Is it OK for elmcity to keep using Delicious the way it has been? If anyone reading this can help me get that question answered I will be grateful.

It got there by way of a new technique I just added to elmcity’s repertoire. Until now, the only way to syndicate events from Facebook through an elmcity hub was the one described here. In that scenario a curator asks an elmcity hub to search Facebook for public events in the hub’s location. This hasn’t worked well. If you use Facebook’s API to search for public events using the term Keene, NH you’ll find events, but only ones that include Keene, NH in the event’s title. Events that mention Keene, NH in the location field are oddly missing.

So until now, if you wanted to use Facebook to promote a public event in Keene, you had to use Keene, NH in the title to get it to work. Being married to me, Luann knows to do that, and her event was already appearing in the hub. But for most people this sort of hack will (and should) be a non-starter.

Even if Facebook’s API worked the way I’d expect, and could find public events by location rather than just by name, it’s not really the right thing. The elmcity model puts event owners in charge of data feeds that hubs (and other consumers) access at specified URLs. It’s nice to know that your public events in Facebook can be found via search. But if you want to syndicate those events through a hub you’d rather publish an explicit feed URL.

It turns out that you can, but in an odd way that requires some explaining and raises important questions about the evolving landscape of online identity. The new elmcity technique relies on the Export link at the bottom of Facebook’s Events page. The label, Export, connotes private use of the iCalendar feed behind that link. The feed is primarily for Facebook users who want to sync their Facebook Events pages to their personal calendars. When you click the link, you’re shown an URL like this:

webcal://www.facebook.com/ical/u.php?uid=652….115&key=AQD…qcT

If you change webcal to http you’ve got one of those special sharing URLs that are public, in the sense that they live on the open web, but also private, in the sense that they’re not discoverable. You wouldn’t use this kind of URL for anything really sensitive. But it can be appropriate for calendars, or friends-and-family photo galleries, or other cases where security-by-obscurity is a reasonable tradeoff.

Syndicating Facebook events

When I fetched the URL for Luann’s Facebook events, by way of her Export link, I had an iCalendar feed that was a mixed bag. It included events to which Luann had been invited by friends; those events were marked private. It also included the event that Luann was promoting; that one was marked public. Ideally Facebook would provide a separate URL for just Luann’s (or any Facebook user’s) public events. There would be nothing secret about that URL, it could be shared freely on the open web.

Lacking such an URL from Facebook, elmcity has to synthesize it. To do so, it filters the private feed to include only events that meet two criteria:

1. They belong to the Facebook user, not to a Facebook friend of that user.

2. They are marked public.

Here are the iCalendar properties that enable such filtering:

ORGANIZER;CN=Luann Udell:MAILTO:luann@luannudell.com
CLASS:PUBLIC

In this example, when I excluded everything in Facebook’s iCalendar feed that wasn’t organized by Luann, and wasn’t public, I was left with a feed containing just one event: Luann’s art opening. That’s the feed I wanted to tell the Keene hub to subscribe to.

My first thought was to use the normal elmcity mechanism for subscribing a hub to calendar feeds. The hub’s curator bookmarks the feed in a designated Delicious account. In this case, there would need to be an extra bit of metadata to enable the hub to filter the feed properly. It could easily find events marked PUBLIC. But how would it know to find Luann’s public events? For that it would need the name of a Facebook user, in this case Luann’s name.

At first blush that seemed easy to solve. The elmcity service uses Delicious tags to convey metadata to hubs. I could define a new convention like so:

The combination of the trusted and ics and feed tags is the normal way a curator tells a hub that the bookmarked URL is an iCalendar feed that the hub should try to process. The who=Luann+Udell tag would be extra metadata telling the hub to restrict a Facebook calendar feed to only the events organized by the named Facebook user.

Problem solved? Unfortunately no. Do you see why not? This mechanism leaks private information. Although the elmcity hub isn’t going to publish anything other than Luann’s public event, her Facebook feed also contains a private event from Judy. Anyone who scans the feeds for the Keene hub would see the unfiltered URL and could find Judy’s private event.

If the architecture of elmcity were more typical, this wouldn’t be a problem. Curators would create accounts at elmcity, they’d log into those accounts to add feeds, the lists of feeds subscribed to by hubs would be hidden from the world. But elmcity does things differently. Hubs are transparent conduits through which public information flows. They reveal their sources. Nothing needs to be hidden, and so nothing is hidden. Curators do their work out in the open. Communities served by elmcity hubs can see how those hubs are constituted.

If Facebook provided a Publish link for public events, along with an Export link for all events, then it could participate directly in elmcity-style calendar syndication. Since Facebook doesn’t offer a Publish link, elmcity needs to receive the Export link from curators by way of some private channel of communication.

Syndicating Facebook events privately

If the architecture of elmcity were more typical, curators would have accounts at elmcity and would use those accounts to send private messages to the service. But again, elmcity does things differently. You shouldn’t have to create an account for everything under the sun. You already have plenty of perfectly good accounts. Why not reuse them?

The relationship between elmcity and Delicious is one example of such reuse. A curator creates a new elmcity hub by designating a Delicious account to control it. If the administrator of the elmcity service deems the proposed new hub legitimate, he or she (so far, just me) tells the service to use that Delicious account to specify the hub’s settings and list its feed URLs.

There’s an analogous relationship between elmcity and Twitter. If a hub’s Delicious settings name a curator’s Twitter account, then Twitter becomes a channel for private messages between the curator and the hub. For example, a curator can send a Twitter direct message to the hub that simply says: start. When the hub receives the start message, it immediately refreshes all the hub’s feeds instead of waiting for the next scheduled refresh.

Until recently that was the only control message that a curator could send to a hub. But now there’s another: add_fb_feed. From my Twitter account I just sent a direct message to the elmcity service’s Twitter account. The message said:

add_fb_feed id=652...115 key=AQD...qcT who=Luann+Udell category=art

Which means the following:

1. Make this Facebook iCalendar URL: http://www.facebook.com/ical/u.php?uid=652….115&key=AQD…qcT

2. Restrict it to public events organized by Luann Udell

3. Use art as the tag for events in this feed

The hub read that message and responded:

elmcity received your add_fb_feed command

And then:

facebook feed successfully added

This isn’t ideal. Curators still need to trust the elmcity service not to disclose URLs sent through the private Twitter channel. Such disclosure could happen in any of the following ways:

• The operator of the elmcity service gives up the data on purpose. I wouldn’t, of course, but it’s theoretically possible.

• The elmcity service gives up the data accidentally. It’s just software; mistakes happen.

• A curator gives up the data either accidentally or on purpose. This risk exists because the elmcity service only has trust relationships with curators. They, in turn, have trust relationships with the contributors who provide calendar feed URLs. Contributors who want to use Facebook as a source for public events that flow through an elmcity hub will give their Export URLs to curators. And curators, accidentally or on purpose, could leak those URLs.

This post is already too long, so I’ll say elsewhere why I think you probably shouldn’t use Facebook as the authoritative source for public events that you want to promote. But if you understand the mechanism I’ve explained here, and if you think the risk/reward tradeoff is acceptable, then you’re welcome to use it. If you’re an elmcity curator who has designated a Twitter account for private communication with the service, here’s the new command you can send to add a Facebook feed from one of your contributors:

add_fb_feed id=UID key=KEY who=USER [category=CATEGORY]

Usage is as follows:

add_fb_feed is a required verb

id=UID is a required parameter. Replace UID with the xxx from uid=xxx in your Facebook Export URL

key=KEY is a required parameter. Replace KEY with the yyy from key=yyy in your Facebook Export URL

who=USER is a required parameter. Replace USER with your Facebook name, using + instead of the space character.

category=CATEGORY is an optional parameter. You can use a single term or a comma-separated list of terms. These will appear as tags on each event flowing from the feed through the hub.

If you send a bogus or missing command verb, or id, or key, you’ll get a Twitter message describing what went wrong.

Given the web’s hybrid nature, how to can we teach people to make best use of this distributed hypermedia system? That’s what I’ve been trying to do, in one way or another, for many years. It’s been a challenge to label and describe the principles I want people to learn and apply. I’ve used the terms computational thinking, Fourth R principles, and most recently Mark Surman’s evocative thinking like the web.

Back in October, at the Traction Software users’ conference, I led a discussion on the theme of observable work in which we brainstormed a list of some principles that people apply when they work well together online. It’s the same list that emerges when I talk about computational thinking, or Fourth R principles, or thinking like the web. Here’s an edited version of the list we put up on the easel that day:

1. Be the authoritative source for your own data

2. Pass by reference not by value

3. Know the difference between structured and unstructured data

4. Create and adopt disciplined naming conventions

5. Push your data to the widest appropriate scope

6. Participate in pub/sub networks as both a publisher and a subscriber

7. Reuse components and services

1. Be the authoritative source for your own data

In the elmcity context, that means regarding your own website, blog, or online calendar as the authoritative source. More broadly, it means publishing facts about yourself, or your organization, to a place on the web that you control, and that is bound in some way to your identity.

Why?

To a large and growing extent, your public identity is what the web knows about your ideas, activities, and relationships. When that knowledge isn’t private, your interests are best served by publishing it to online spaces that you control and use for the purpose.

Related

Mastering your own search index, Hosted lifebits

2. Pass by reference rather than by value

In the case of calendar events, you’re passing by value when you send copies of your data to event sites in email, or when you log into an events site and recopy data that you’ve already written down for yourself and published on your own site.

You’re passing by reference when you publish the URL of your calendar feed and invite people and services to subscribe to your feed at that URL.

Other examples include sending somebody a link to an article instead of a copy of the article, or uploading a file to DropBox and sharing the URL.

Why?

Nobody else cares about your data as much as you do. If other people and other systems source your data from a canonical URL that you advertise and control, then they will always get data that’s as timely and accurate as you care to make it.

Also, when you pass by reference you’re enabling reuse (see 7 below). The resources you publish can be recombined, by you and by others, with other resources published by you and by others.

Finally, a canonical URL helps you measure how the web reacts to your data. If the URL is cited elsewhere you can discover those citations, and you can evaluate the context that surrounds them.

Related

The principle of indirection, Hyperlinks matter

3. Know the difference between unstructured and structured data

When you create an events page on your website, and the calendar on that page is an HTML file or a PDF file, you’re posting unstructured data. This is information that people can read and print, and it’s fine for that purpose. But it’s not data that networked computers can process.

When you publish an iCalendar feed in addition to your HTML- or PDF-based calendar, you’re publishing data that machines can work with.

Perhaps the most familiar example is your blog, if you have one. Your blog publishing software creates an HTML page for people to read. But at the same time it creates an RSS or Atom feed that enables feedreaders, or blog aggregation services, to automatically collect your entries and merge them with entries from other blogs.

Why?

When you publish an iCalendar feed in addition to your HTML- or PDF-based calendar, you’re publishing data that machines can work with.

The web is a human/machine hybrid. If you contribute data in formats useful only to people, you sacrifice the network effects that the machines can promote. If you also contribute in formats the machines understand, they can share your stuff amongst themselves, convey it to more people than you can reach through word-of-mouth human networks, and enable hybrid human/machine intelligence to work with it.

Related

The laws of information chemistry, Developing intuitions about data

4. Create and adopt disciplined naming conventions

When people publish calendars into elmcity hubs, they can assign unique and meaningful URLs and/or tags to each event they publish. And they can collaborate with curators of hubs to use tag vocabularies that define virtual collections of events.

The same strategies work in all web contexts. Most familiar is the first order of business at every conference attended by web thinkers: “The tag for this conference is ______.” When people agree to use common names in shared data spaces, effects like aggregation, routing, and targeted search require no special software.

Why?

The web’s supply of unique names (e.g., URLs, tags) is infinite. The namespace that you can control, by choosing URLs and tags for the things you post, is smaller but still infinite. Web thinkers use thoughtful, rigorous naming conventions to manage their own personal information and, at the same time, to enable network effects in shared data spaces.

Related

Heds, deks, and ledes, The power of informal contracts, Permalinks and hashtags for city council agenda items, Scribbling in the margins of iCalendar

5. Push your data to the widest appropriate scope

When you speak in electronic spaces you can address audiences at varying scopes. An email message addresses one or several people; a blog post on a company intranet can address the whole company; a blog post on the public web can address the whole world. Web thinkers know that keystrokes invested to capture and transmit knowledge will pay the highest dividends when routed to the widest appropriate scope.

The elmcity example: a public calendar of events can be managed in what is notionally a personal calendar application, say, Google Calendar or Outlook, but one that can post data to a public URL.

For bloggers, this principle governs the choice to explain what you think, learn, and do on your public blog (when appropriate) rather than in private communication.

Why?

Unless confidentiality precludes the choice, web thinkers prefer shared data spaces to private ones because they enable directed or serendipitous discovery and ad-hoc collaboration.

Related

Too busy to blog? Count your keystrokes

6. Participate in pub/sub networks as both a publisher and a subscriber

Our everyday calendar programs are, in blog parlance, both feed publishers and feed readers. Individuals and organizations can publish their own feeds to the web of calendar data while at the same time subscribing to others’ feeds. On a larger scale, an elmcity hub subscribes to a set of feeds, and in turn publishes a feed to which other individuals (or hubs) can subscribe.

Why?

The blog ecosystem is the best example of pub/sub syndication among heterogeneous endpoints through intermediary services. Similar effects can happen in social media, and they happen in ways that people find easier to understand, but they happen within silos: Facebook, Twitter. Web thinkers know that standard protocols and formats enable syndication that crosses silos and supports the most open kinds of collaboration.

Related

Personal data stores and pub/sub networks

7. Reuse components and services

In the elmcity context, calendar programs are used in several complementary ways. They combine personal information management (e.g., keeping track of your own organization’s public calendar) with public information management (e.g., publishing the calendar).

In another sense they serve the needs of humans who read those calendars on the web while also supporting mechanical services (like elmcity) that subscribe to and syndicate the calendars.

In general, a reusable web resource is:

1. Effectively named
2. Properly structured
3. Densely interconnected (linked) both within and beyond itself
4. Appropriately scoped

Why?

The web’s “small pieces loosely joined” architecture echoes what in another era we called the Unix philosophy. Web thinkers design reusable parts, and also reuse such parts where possible, because they know that the web both embodies and rewards this strategy.

Related

How will the elmcity service scale? Like the web!, How to manage private and public calendars together

A few weeks ago, my in-laws were visiting. She is a pretty famous book author and we were talking about how technology could bring value to her workflow (she is not flat out an IT luddite but close enough).

She just created her first web site and asked me how she could promote it on search engines (classic newbie SEO question). She was not interested in the mechanics at all, she just wanted more exposure.

We talked about Twitter and about how publishers and authors use it to promote themselves and engage their audiences. She thought all this was very “Hollywood” and not her style at all. But I showed her that you don’t need to use Twitter that way, you can just mine it for your ego network. Then I explained how I set up all sorts of traps around the web, with newsfeeds, and how I use Google Reader to aggregate them all for me.

I showed her right there and then. Searched for her new book name on Twitter, clicked on the RSS feed, did the same on Google blog search, Google news search, and voila, her personal PR aggregation network was born.

She was completely blown away. She didn’t know any of this was even remotely possible, yet, once explained, it make perfect sense. It’s like having personal agents watching everything that goes on and sending you the information. Email versus RSS doesn’t make any difference to her. As long as she has a place to go and check out what others say about her, she’s happy.

My take: no tech-unsavvy person thinks it reasonable to have a personal agent that does, for individuals and for free, what gigantic organizations struggle to do every day.

The fact that Google can search 15 billion pages in milliseconds doesn’t faze them as much. If librarians can do it, so does Google. Big deal.

But personal agents constantly working in the cloud for you? It doesn’t even show up in the realm of possibilities.

If Stefano’s in-law were on Facebook, of course, she’d be getting a sense of what it’s like to have one of those agents in the cloud. Her activity stream would magically be visible to friends, and their reactions to it would magically be visible to her. That’s why I often say, nowadays, that Facebook is a great set of training wheels for the pub/sub network.

But Facebook isn’t, yet, a place where people can learn how to publish data that syndicates beyond Facebook. It’s possible, as I discussed in Heds, deks, and ledes, to post public events on Facebook in a way that can be discovered by an elmcity hub or by some other agent. If you don’t know such agents can and do exist, though, you’ll never stop to think about whether they’re actually finding your data — and if not, how to make sure that they do.

One of the key points embedded in Stefano’s parable is that his in-law didn’t have to do anything special in order to be able to find the web’s reaction to her book. To the extent that her name and the name of her book are out there and indexed, they provide good-enough hooks for search aggregation.

Over time, of course, the efficacy of these searches will decay. I’ve watched this happen with my own name. Years back, my stuff was pretty much the only stuff that a search for Udell would find. (There was even a time when the first Jon on Google was me, not Jon Stewart!) Then my wife began showing up, along with a whole bunch of other Udells. So I tuned my filters to Jon Udell, and they work better for now, but there are other Jon Udells and it’s only a matter of time before that namespace gets cluttered too.

In order to reliably find stuff about me, I need filters tuned to aspects of my identity: my domain name, my Twitter handle. Eventually Stefano’s in-law may reach the same conclusion. She may realize that posting to her website is more than a way to share her thoughts with the world. It also enables the world to react to her posts in ways she can, in turn, discover. At that point she may start to see why it’s important to actively colonize parts of the web that are, or can be, bound to aspects of her identity.

The elmcity project, similarly, invites promoters of public events — and communities at large — to colonize the web in ways bound to those individual or group identities. When you produce a calendar feed that flows through an elmcity hub, you’re not just helping to populate that hub. That feed is attached to your own site and, in theory, is directly discoverable there. In practice, though, there are aren’t yet good methods of discovery. We don’t yet have, for iCalendar, an autodiscovery mechanism like the one we have for RSS. That’d be easy enough, as Mark McClaren suggests:

Love it or hate it, iCalendar is the pervasive calendaring format. If we can enable RSS autodiscovery then why not do the same with iCalendar feeds. Adding one line of code would make it easier for people/machines to subscribe to an iCalendar feed.

<link rel="alternate" type="text/calendar"
  title="iCalendar feed for example.com"
  href="calendar.ics" />

It would also be really helpful to be able to bind locations to events in a discoverable way. To that end I’ve recently enhanced the HTML rendering of elmcity hubs. Now they include what Google calls rich snippets, using the RDFa-style markup documented here. The snippets include latitude and longitude coordinates derived in one of two ways:

1. Per-event. There are several ways that an event can show up bearing latitude/longitude values. The vast majority of such events will be those coming from Eventful and Upcoming, both of which services provide lat/lon values via their APIs. There’s also a GEO property defined for iCalendar, and some iCalendar producers use it to geocode events.

2. Per-hub. Although most iCalendar producers don’t use the per-event GEO property, elmcity hubs know their own locations. So events that lack specific lat/lon coordinates inherit the locations of their hubs.

It’s going to be a while yet until folks like Stefano’s book-writing in-law start to realize they can, as Kingsley Idehen nicely puts it, master their own seach indexes. But sooner or later they’ll realize that it’s possible. Likewise, it’ll be a while yet until promoters of public events realize that the event data they push to their websites can not only feed pub/sub networks, but can also feed location-aware search engines. I’m a patient man, though, and I do expect the seeds I’m planting to grow and eventually bear fruit.

A bicyclist riding along Old Homestead Highway was hit by a vehicle Friday evening.

At about 6:43 p.m. Swanzey Police and Fire Department responded to a reported hit-and-run accident on Route 32.

The vehicle was described as a white SUV, possibly a Chevy Blazer, with a black roof rack. It’s missing its passenger-side mirror as a result of the accident, according to Cpl. Robert Eccleston of the Swanzey police.

The cyclist suffered serious injuries and was transported to Cheshire Medical Center/Dartmouth-Hitchcock Keene.

A couple of years ago it was me. I got sideswiped on a bike ride in another part of the county. In that case too, the impact also broke off the passenger-side mirror. Luckily I only suffered a bruised leg. According to a follow-up report, this cyclist suffered “skull fractures on the left side of his head, where his helmet hit the pavement, a broken shoulder and severe road rash.”

When it happened to me, I was furious for weeks. Every time I saw a sedan similar to the one that knocked me off my bike I looked for the telltale missing passenger-side mirror. And I formed a clear idea of a product that might have prevented the hit-and-run, or failing that, nabbed the perpetrator. It’s a pair of bicycle-mounted cameras, front and rear, that trigger on approaching traffic and take sequences of shots that can identify approaching vehicles.

Here’s why I imagine this could work. I don’t know about yesterday’s hit-and-run, but in my case it didn’t feel like an accident. We were the only two vehicles on the road. There was plenty of room for the car to give me wide berth. But some motorists like to hassle cyclists verbally, and once in a while that escalates to a cat-and-mouse game. That’s a game people these people play because they think they can get away with it. There’s no expectation that the sideswiped cyclist will be able to prove that it happened, or capture the identity of the car. In my case, when I jumped to my feet after tumbling along the roadside, only to see the car speeding over the top of the next hill, I remember thinking: “You bastard, if I only had your license plate number you would regret this.”

Defensive surveillance isn’t just a capability that cyclists need, of course. It makes sense for motorists to identify and record oncoming traffic too. But car-on-car violence is a game played on a level field. Car-on-bike violence is so unequal that I’ll jump at any advantage I can get.

Does the product I imagine already exist? Maybe, but I don’t think so. There are obviously scads of cheap helmet- or bike-mountable cameras. What I’m looking for, though, is one that’s optimized for defensive surveillance. I think that means a gadget that senses oncoming traffic, and then shoots sequences of high-resolution stills. Ideally it’d come with two pairs of mounts. One pair would be fitted to my bike’s handlebar and seat. The other pair would be fitted to my car’s dashboard and rear deck. For extra credit, the car would keep the cameras charged so they’re always ready to defend the bike.

PS: Meanwhile, my low-tech solution is a helmet-mounted rear view mirror. I have always used one, and can now scarcely imagine what it used to be like to have to crane my head around — and wobble my bike — in order to see what’s behind me. With a helmet mirror, situational awareness only requires rapid eye flicks that become an automatic habit. Obviously the habit wasn’t fully automatic, but after the incident a couple of years ago I’m even more vigilant. I watch every car that approaches from the rear, and am always mentally preparing a dive into the ditch.

Update: From Crain’s:

The Senate legislation would push most of the $615 trillion in over-the-counter derivatives onto regulated exchanges or similar electronic systems, a measure that would make it easier for the market and regulators to track the trades.

Really? Well OK then! Fingers crossed.

Ten bucks wasn’t enough to get them as far as they said they needed to go, though. And later I got to thinking about how we might have created enough trust, in an ad-hoc way, for me to make a short-term loan of, say, $50. It’s an interesting thought experiment. I wonder what solutions you can imagine? Here are a few that occurred to me.

Web identity. Given a web connection, I could have searched for the couple’s names, found their web footprints, and verified that their photographs, locations, and other attributes matched what they claimed.

Six degrees of separation. If we could trace our connection through social network space, that might be enough. It might even be possible to do that with voice calls, but with a web connection it could be almost trivial.

PayPal. Given a web connection, we could have brought up a browser and done a PayPal transaction. In that case I wouldn’t even be making a loan, I’d know that the funds had been transferred before handing over cash.

Losing my wallet while traveling is a nightmare scenario for me. It’s never happened but I dread the thought. I hate being so dependent on documents that I carry around in a wallet that could easily be lost or stolen.

Those documents embody claims made on my behalf by identity providers that we have all agreed to trust. That arrangement became necessary when society grew beyond what interpersonal trust could scale out to support. And it will remain necessary. But as voice and data connectivity become ubiquitous, and as interpersonal trust scales out in ways it never could before, I wonder if we’ll see a re-emergence of pre-bureaucratic modes of identity.

Scott writes:

I’m a geek, I like tools and I solve problems in my own niche way.

That statement, along with the essay’s tagline — …Tips for the Geek-Minded Person — has been bothering me ever since I read it. Why is it geeky to marshal the best available data? Why is it geeky to use that data to improve your interaction with people and processes?

My Wikipedia page includes this sentence:

Udell has said, “I’m often described as a leading-edge alpha geek, and that’s fair”. ¹

I did say that, it’s true. But I’ve come to regret that I did. For a while I thought that was because geek was once defined primarily as a carnival freak. That’s changed, of course. Nowadays the primary senses of the word are obsessive technical enthusiasm and social awkwardness. Which is better than being somebody who bites the heads off chickens. But it’s still not how I want to identify myself. Much more importantly, it’s not how I want the world to identify the highest and best principles of geek identity and culture.

Fluency with digital tools and techniques shouldn’t be a badge of membership in a separate tribe. In conversations with Jeannette Wing and Joan Peckham I’ve explored the idea that what they and others call computational thinking is a form of literacy that needs to become a fourth ‘R’ along with Reading, Writing, and Arithmetic.

The term computational thinking is itself, of course, a problem. In comments here, several folks suggested systems thinking which seems better.

Here’s a nice example of that kind of thinking, from Scott’s essay:

#3 Make their job easy

Speak their language and tell them what they can do to get you out of their hair. Refer to flights by number when calling reservations, it saves huge amounts of time. For example, today I called United and I said:

“Hi, I’m on delayed United 686 to LGA from Chicago. Can you get me on standby on United 680?”

Simple and sweet. I noted that UA680 was the FIRST of the 6 flights delayed and the next one to leave. I made a simple, clear request that was easy to grant. I told them where I was, what happened, and what I needed all in one breath. You want to ask questions where the easiest answer is “Sure!”

I see two related kinds of systems thinking at work here. One engages with an information system in order to marshal data. Another engages with a business process — and with the people who implement that process — in a way that leverages the data, reduces process friction, and also reduces interpersonal friction.

These are basic life skills that everyone should want to master. If we taught them broadly, and if everyone learned them, then this sort of mastery wouldn’t attract the geek label. But we don’t teach these skills broadly, most people don’t learn them, and the language we use isn’t our friend. If systems thinking is geeky then only geeks will be systems thinkers. We can’t afford for that to be true. We need everyone to be a systems thinker.

¹ Actually I’d say that Scott Hanselman is a leading-edge alpha geek. I am, at best, a trailing-edge beta or gamma geek. But if someone were to remove the word entirely from my Wikipedia page, I’d be fine with that. I no longer want to be labeled as any kind of geek.

Inspired by a PHP solution to the problem, Scott set out to replicate it for ASP.NET.

But then I realized that a server-side solution wasn’t really necessary.

Could I do it all on the client side? This way anyone could add this feature to their site, regardless of their server-side choice.

One next step, as Scott points out, is to add geolocation so the list of kids you see will be more relevant to you. But there are lots of ways to contextualize that list based on aspects of your identity. And this is what Kynetx applications do: Contextualize your experience of the web based on aspects of your identity.

My own interest in this idea dates back to the LibraryLookup project, which was an early demonstration of the power of client-driven contextualization. It evolved from a bookmarklet to a browser plug-in, but then stalled there for lack of a ubiquitous client-side technology.

Now there is: jQuery. What Scott’s example shows, as do all Kynetx applications, is that we’re ready to make clients more equal partners in the dance of the web. Among other things, this possibility raises horny issues about the control of content — issues that I explored in a 2005 screencast.

But there’s also a deep connection between Phil’s work and the ongoing saga of digital identity. Phil wrote a book on that subject, and has been a key organizer of the Internet Identity Workshop. When he started Kynetx he wasn’t really thinking about a tie-in to Information Cards and the identity metasystem. But the connection emerged organically.

In a Kynetx-enhanced version of the Missing Kids 404 Page application, your browser would present selected aspects of your identity to the services that provide the data, and a Kynetx application would personalize that data in ways meaningful to you.

The Internet began as a network of peers. That arrangement didn’t last long, and there have been several efforts to restore the original symmetry. In the early 2000s, during Napster’s heyday, there was a flurry of interest in peer-to-peer architectures. Thanks to today’s more capable and more standardized browsers, we’re seeing a new wave of interest. I’m looking foward to hanging out at the Kynetx conference and meeting folks who are riding that wave.

Imagine a web service that enables parents to find available babysitters. A compromise would disastrously reveal vulnerable households where parents are absent and teenage girls are present. Translucency, in this case, means encrypting sensitive data (identities of parents, identities and schedules of babysitters) so that it is hidden even from the database itself, while yet enabling the two parties (parents, babysitters) to rendezvous.

Fast forwarding to 2009, here’s a current headline from InfoWorld: Microsoft adds access controls for SQL Azure online database. The article doesn’t say so, but this is database translucency in action.

The 2009 version of the babysitter example appears at 37:45 in this PDC session, where Dave Campbell and Rahul Auradkur discuss, and also show, a translucent pharmaceutical reagent marketplace. Dave Campbell spells out the scenario:

Pharma companies see reagents as being pre-competitive. They don’t compete at that level, and they’re willing to sell these reagents to one another, as long nobody can see what’s being bought and sold. That’s the controlled trust we need to set up.

The trick is accomplished by means of encryption and careful separation of concerns. Access policies are isolated from data storage, capable of federation, and auditable by trusted intermediaries.

This is exciting new territory. Historically, we’ve always assumed that the operator of an online information system has complete access to the data in that service. Translucency turns that assumption on its head, and leads to entirely new service design patterns. To implement those patterns requires more than just a database in the cloud. You also need a coordinated suite of supporting services for identity, access control, auditing, and more. Azure, as it becomes one provider of such services, will help make translucency a practical reality.

It wasn’t perfect. Sometimes the passwords it generated wouldn’t meet a site’s requirements. And sometimes the login domain name would vary, which broke the scheme. But it introduced me to two powerful — and related — ideas. JavaScript could turn your browser into a programmable cryptographic engine. And that engine could be used to implement protocols that relied on cryptography but transmitted no secrets over the wire.

To my way of thinking, that’s a killer combination. For years I’ve been using Bruce Schneier’s Password Safe, a Windows program that keeps my passwords in an encrypted store. There are many such programs, another example being 1Password for the Mac. This kind of app lives on your computer and talks to a local data store. That means it’s cumbersome to move the app and your data from one of your machines to another. And you can’t use it online, say from a public machine at the library or a friend’s computer.

Imagine a web application that would encrypt your credentials and store them in the cloud. It would deliver that encrypted store to any browser you happen to be using, along with a JavaScript engine that could decrypt it, display your credentials, and even use them to automatically log you onto any of your password-protected services. You’d trust it because its cryptographic code would be available for security pros to validate.

I’ve wanted this solution for a long time. Now I have it: Clipperz. My guest for this week’s Innovators show is Marco Barulli, founder and CEO of Clipperz, which he describes as a zero-knowledge web application. What Clipperz has zero knowledge of is you and your data. It just connects you with your data, on terms that you control, in a way that reminds me of Peter Wayner’s concept of translucent databases.

Clipperz is immediately useful to all of us who struggle to manage our growing collections of online credentials, But it’s also a great example of an important design principle. We reflexively build services that identity users and retain all kinds of information about them. Often we need such knowledge, but it’s a liability for the operators of services that store it, and a risk for users of those services. If it’s feasible not to know, we can embrace that constraint and achieve powerful effects.

For example, friends and family used to be impressed by the fact that I was the top result in Google for my first name — and then second to Jon Stewart for a long while, until I had to reboot my InfoWorld archive. Why? Just because I’ve projected a large surface area of searchable documents whose titles include the trigram jon.

An example of a far more notable person than me is Glenn Fine, who was in my grade in junior high school and is now Inspector General for the Department of Justice. You won’t find him anywhere near the top of a search for his first name because Inspectors General don’t (yet) project a large surface area of documents onto the web.

To place my newfound Wikipedia notability into a similar context, I wanted to show people how these lists of notable inhabitants are made. I figured the person who made the change is somebody who knows of my work, because I’ve written about it so much online, and who is inclined to edit Wikipedia, which correlates with an interest in my work.

I wanted to illustrate exactly who, when, and how, so I went to Wikipedia with the confident expectation that it would be easy to answer those questions.

Surprisingly, it wasn’t. I guess I haven’t really tried searching revision histories in Wikipedia before, but in this case and a few others I’ve tried lately, it seems quite difficult to pinpoint the author of a change.

For example, on Twitter I asked:

Wikipedia: “The term ‘Web 2.0′ was coined by Darcy DiNucci in 1999.” Added when, by whom? WikiBlame seems an ineffective way to find out.

@bazzargh replied: Robert Gehl. http://bit.ly/46r1a

Thanks. By the way, how’d you do that?

switch to 500 view in history, then rough bisection from oldest. Couple of minutes; used this a lot to find long-lived vandalism.

if older, I progressively back off 2..4..8… pages through this. In this case though, there was a clueful log message!

That’s pretty much what I’ve found myself doing when trying to track down changes, so I was glad to know it wasn’t just me. But this highlights an important point about transparency: It’s all relative.

One of the reasons we think of government as opaque is that while records may be notionally public, it takes time, effort, and skill to visit city hall, dig through them, and find what you’re looking for.

I have always regarded Wikipedia as an extreme counter-example. And that’s true. It is radically transparent. You can ultimately find out exactly how any statement in any article came to be. You may not be able to correlate the author’s pseudonym to a real-world identity, but you can evaluate that author’s corpus and reputation within the context of Wikipedia.

And yet, the ability to do this spelunking requires more time, effort, and skill than most people possess. Although I’m reluctant to deflate my status as a notable inhabitant of Keene, I wish it were easier for people who read that to also find out what it does — and doesn’t — mean.

1. Choose an identifier for your new hub. For example: ypsicals.

2. Choose an identifier for yourself. This can be a Facebook name (or id), a Twitter name, a Gmail address, or a Windows Live email address.

3. Notify the elmcity administrator that you’re ready to start your new hub.

4. When you receive confirmation that the hub has started, visit http://elmcity.cloudapp.net and log in using your Facebook, Twitter, Gmail, or Windows Live identity.

There are a number of moving parts in this scenario. On the back end, Kynetx provides a rules engine that decides how to rewrite a page based on the context of the user’s “web episode” and the user’s membership in an organization like AAA. Membership is asserted by an Information Card that the user installs, then presents on request to a browser extension. It asks the Kynetx service for a chunk of page-modifying JavaScript, then runs that code locally to effect the change specified by the rule.

If you’ve followed the Internet identity saga — a story that Phil has helped to write, as author of a book on digital identity and as an organizer of the Internet Identity Workshop — you’ll be thrilled to see that the Kynetx system is responsible for the minting and real-word use of Information Cards. As Phil explains in this interview, the cards as currently used convey no extra information, they merely signify membership. Still, it’s great to see this key technology finally percolate out into the mainstream.

Kynetx will mainly serve companies that want to solidify and enhance high-value relationships with customers by means of “permission-based context management.” Refreshingly, the Kynetx wiki qualifies that definition in a way that will make Doc Searls smile:

The following anti-lexicon contains words and concepts that Kynetx doesn’t use:

• exploit – while opportunities might be exploited, people never should be.

• eyeballs – we’re not doing optometry

• target – you target enemies, not customers.

Near the end of the interview, Phil refers explicitly to Doc’s VRM (Vendor Relationship Management) campaign:

We see ourselves as plumbing for VRM. For example, we’re putting together a green choice card. If you install it, as you search around the web it will show you which companies have been ranked well or poorly in terms of social responsibility. Right now it’s just a demo, and we don’t have great data, but suppose we did, and there were enough of those cards out there, and Constellation Brands was determined by Fortune Magazine to be the least socially responsible company in 2008. If every time a cardholder found a Constellation product on Google there was a little icon indicating that, and there were a lot of people with the card, you could change the company’s behavior. They’d want to get the icon off that page.

It’s a fascinating notion, and it leads to an issue that I should’ve raised with Phil in the interview but will raise here instead. A couple of years ago, during my period of infatuation with Greasemonkey, I made a 4-minute screencast entitled Content, services, and the yin-yang of intermediation. At the time, I’d just invented a Greasemonkey-enabled version of LibraryLookup that was more aggressive than the standard bookmarklet version.

With the standard version, you click a bookmarklet while on an Amazon page, and a query against your local library pops up in a window. With the Greasemonkey-enhanced version, the Amazon page itself is rewritten to say:

“Hey! This book’s available at the Keene Public Library!”

Or:

“Due back at the Keene Public Library on March 28.”

But does the user of a web-based service have the right to modify pages in these ways? The screencast ponders that question. Three years ago there wasn’t enough client-side page rewriting going on to raise that question in a big way, and I guess there still isn’t, but now that jQuery is making the capability broadly available it’s bound to come up.

There’s a continuum of ways in which I can modify a web page in a browser, ranging from font enlargement to translation to contexual overlays. I wouldn’t draw a line anywhere along that continuum. It seems to me that I’m entitled to view the world through any lens I choose.

This doesn’t only apply to my view of the virtual world, by the way. It will apply to my view of the physical world too. We don’t yet have magic glasses that overlay web prices on shelf items, or web reputations on store signage, but someday we will.

I can’t see how I could be prevented from creating a heads-up display — for realspace or cyberspace — that’s advantageous to me. But I’ve got a hunch that those magic glasses are going to be controversial.

There’s a crucial difference between a) committing my words to Doug’s blog, and b) committing my words to my own lifebits stream and then syndicating them to Doug’s blog. We don’t see it very clearly yet because we lack the mechanism for b).

I can kinda get the effect of syndication by referring to Doug’s blog entry from mine, and hoping that his blog engine will notice and acknowledge. But a truly syndication-oriented mechanism would imply that I publish in my own space, and then — in Doug’s space — actively subscribe back to myself. To explicitly comment on Doug’s entry, in other words, I don’t type words into his comment form. I create a subscription associated with my identity (as a conventional comment always is) that points back to my feed.

Let’s consider Doug’s point #4: “You determine if/when/how this data is accessed, the terms of use and the revocation of the license.” If I comment on Doug’s blog, I can hope for ex post facto control of my words, but whatever agreement may be (tacitly or explicitly) in place, the architecture doesn’t support that control. I may or may not be able to revise or extend my remarks. And Doug can certainly revise, extend, or delete — it’s his blog.

If I syndicate to Doug’s blog, there is still only a hope of ex post fact control, not a guarantee. But the architecture is at least aligned in my favor. The effort I invest in writing on Doug’s blog, or a bunch of other blogs, is preserved. I can archive, organize, and search all my stuff. I don’t need to depend on services Doug’s blog may or may not offer to find out who is reading and reacting to my stuff. And if I want to withdraw my comment, I just revoke the permission I gave Doug’s blog service to syndicate from mine.

Realistically, that revocation won’t erase my contribution to Doug’s blog. My words may have been quoted there, in other comments, and the mixing process dilutes control — which I argue is a feature, not a bug. But if the default is to syndicate by reference, rather than by value, the architecture favors the kind of control we want.

To clarify what I mean by favoring the right kind of control, let’s switch to a medical information scenario. Recently I had a dental xray. The image lives on the dentist’s hard drive. I want it to work differently. When I show up at the dentist’s office, I want to give the xray technician a token that grants her machine access to my lifebits store. The machine publishes the image to my store. I, in turn, agree to syndicate the image back to the dentist — maybe to copy, but maybe only to view.

One interesting benefit of this arrangement is that I’m decoupling dental service from image storage service. Maybe I’ll just turn around and reconnect them, because maybe I’d rather just let the dentist bundle those services. But when I interpolate my lifebits store into the pipeline, I guarantee portability to another dentist.

Another benefit is clarity of ownership and syndication rights. My lifebits store will have a management service where I declare, review, and adjust all of the syndication relationships between my lifebits streams and the services they participate in. And this management service can not only implement my ownership and syndication policies, it can announce them to the world. It can be the place where I say who gets to do what with my stuff. Some of those policy assertions will be private, but many will be public. Ultimately, again, there is no guarantee of ex post facto control. But if you violate my terms, it will be easier for me, or anyone, to determine that you have done so.

PS: Coincidentally, or maybe not, Doug was my guest on last week’s Innovators show. The topic was “Oslo”. But the context was our shared passion for figuring out how computers, information systems, and networks can more easily and more faithfully express the intentions of the people who own, operate, and inhabit them.