How Thali could make the Smallest Federated Wiki even smaller

Thanks to my friend Mike Caulfield, an educational technologist who’s been digging into Ward Cunningham’s Smallest Federated Wiki, I’ve now got a much clearer idea of how SFW and Thali could play together and why they should.

Mike’s recent series on SFW is the best review and analysis of Ward’s newest creation that I’ve seen:

I had dipped a toe into the SFW water but there’s a learning curve and Mike climbed it before I could. Today he jumpstarted me by setting me up with a node of an SFW federation he’s hosting on AWS. Here I am participating in a wiki federation with some friends in the ed-tech tribe. We are able to do this because Mike provisioned SFW instances for each of us.

What’s the Thali connection? Well, in the first few seconds of you see Mike provisioning a node in a federation he’s hosting on AWS. That’s the minimum bar for SFW: you need an instance of the server. Most people can’t or won’t leap over that bar.

But the server’s a pretty small piece of the pie. Most of SFW runs in the browser. There’s a lot there, and it’s well-architected for growth.

A server implementation for Thali would enable lots more people to create and participate in Wiki federations, by running SFW on their own devices and syncing opportunistically with peers on friends’ devices. Since the existing Sinatra-based SFW is CouchDB-aware, Thali — based on Couchbase Lite — should provide a comfortable home.

Why would people want to use SFW? Mike’s posts and screencasts point to a world in which GitHub-like collaboration breaks out of the geek ghetto and becomes a natural way for all kinds of teachers and learners to collaborate.

Ward points to that possibility and others in a series of SFW screencasts at I’d seen a few, tonight I went back and watched the rest. Some highlights:

On forking and comparing

An inline calculator plugin (in 25 lines of CoffeeScript!)

Visualization of in-page data

These demos really capture the idea of the universal canvas ( that I’ve dreamed of for a long time.

My 2006 InfoWorld article said, by the way,

Here’s the best definition of the universal canvas: ‘Most people would prefer a single, unified environment that adapts to whichever environment they are working in, moves transparently between local and remote services and applications, and is largely device-independent — a kind of universal canvas for the Internet Age.’

You might expect to find that definition in a Google white paper from 2006. Ironically, it comes from a Microsoft white paper from 2000, announcing a “Next Generation Internet” initiative called .NET.

You never know how things will turn out.

Mapping the decentralization movement

“Right now we’re experiencing a moment of maximum centralization,” says Scott Rosenberg in his introduction to a new effort that combines “a tech-industry beat I will cover; a cultural investigation and conversation I will undertake; and a personal-publishing venture I am kicking off now.”

We’ve been here before. The Internet was a peer-to-peer network until it wasn’t. Likewise the Web. Some have forgotten, and most never knew, that Tim Berners-Lee’s original browser could write and publish as well as read pages. By the early 2000s the pendulum had swung so far toward centralization that, as it began to swing back, we called the “two-way web” one of the pillars of “Web 2.0.” Personal publishing flourished for a while, then the pendulum swung again toward centralized social media. If Scott’s right, and I hope he is, the pendulum is about to swing back toward a more distributed Web.

Thali is one project moving in that direction, there are many others. When we compared notes with Jeremie Miller the other day, he pointed us to a long list of fellow travelers. Another observer, Doc Searls, periodically issues updates with pointers to related (and some of the same) efforts.

It behooves all of us to sort out how these efforts are similar or different along various axes. Some are peer-to-peer, others not. Some bind identities to public keys, others don’t. Some skew toward messaging and social networking, others toward bulk data exchange or publishing. Some consider themselves personal data stores, others don’t. Many are “friend-to-friend” networks with peer-to-peer trust models, some aren’t. There are platforms, protocols, overlay networks, and apps in the mix.

In order to reason about these axes of comparison I loaded up a bunch of links into Pinboard, made a common tag (redecentralize) to unite all the links related to this exploration, and began tagging. Here’s what I’ve got at so far:

What else belongs on this list? What are core attributes? What are the best axes along which to compare? The tag cloud is suggestive but it’s only my lens on the list, I’d love to see other lenses applied to the same (evolving) list.

Note that Pinboard (as with long ago) such lenses can be applied — and in a decentralized way! You could import my redecentralize feed into your own Pinboard account and tag the links according to your world view. We could compare one anothers’ views, and see a combined view at While that’s a very cool way to do collaborative mind-mapping, it’s not likely to happen in this case. But comments here (or elsewhere) will be welcome.

A world without hearsay

If you received email from me in the early 2000s, it would have arrived with an attachment I routinely added to my messages. The attachment was my digital signature, the output of an algorithm that combined my message with the private half of my cryptographic key pair. If you had acquired my public key as part of a prior communication, and if your email client supported the protocol, you were assured that the message had been “signed” by me. Since those two conditions rarely applied, though, you were more likely to be puzzled or annoyed.

Why did I do this? As I look back, I think it had a lot to do with my experience in the early blogosphere. Back then blogs didn’t support comments. To comment on something you wrote, I’d write a blog post referring to your post. You could reply in the same way. The conversational thread didn’t exist in any one place, but in practice links wove the discussion together well enough. And because all our writing appeared on our own blogs, we owned our words. Discourse was typically much more civil than in discussion forums, or in the comment areas that blogs later evolved.

I liked the idea that my digital output was bound in some way to my identity. In the case of blogging, that identity was associated with a website that I controlled. Why not extend that idea to email? In that case, my identity was associated with a key pair, the private half of which I controlled. Signing messages was a way to say: “I own and stand behind these words.” And to say: “You should distrust a message ‘from’ me that isn’t properly signed.”

When I abandoned my digital signature experiment I chalked it up to a failure of technology adoption. It was, I thought, a good idea that never took off because people didn’t understand why it was a good idea, or because popular software didn’t make it accessible enough.

Now a hugely popular email program, Gmail, is about to make the idea more accessible than it’s ever been. Google is preparing a Chrome extension called End-to-End for encrypting (and signing) email[1]. I should rejoice! But Yaron Goland thinks otherwise. He argues here that routinely binding our identities to our messages is a really bad idea.

Imagine, for a moment, what your world would look like if every time you had a conversation with someone a permanent record was made of the conversation. The record would be fully authenticated and suitable for use in the court of public opinion and/or law.

In this world our everyday lives, our conversations, our exchanges, with anyone about anything become little permanent records that follow us around forever.

This is exactly the world we create with technologies like S/MIME and PGP Mail. Or, more generally, the world we create when we use digital signatures. A digital signature is intended to be an authenticator, a way for someone other than us to prove that we did/said something. When we use digital signatures for momentous things that should be on the public record, like mortgage documents perhaps, then they serve a good purpose. But with PGP Mail we suddenly sign… well… everything[2]. It’s like having a notary public walking behind you all day long stamping every statement, note, mail, etc. as provably and irrevocably yours.

I don’t think we want such records to exist. I think we want a much more ephemeral world where the bulk of what we do just quietly vanishes into the ether leaving as little of a trail as possible. The open source experiment I’ve spent the last year or so working on (and why I haven’t been blogging much, I’ve been insanely busy) is called Thali and we are trying to build that ephemeral world.

Yaron calls the dystopian vision he conjures “a world without hearsay” and Thali rejects it. When you communicate with a Thali peer, you and the peer strongly authenticate one another. But the data you exchange bears no trace of those identities. At least not by default. Thali applications will, of course, often need to add markers of identity to the documents they exchange. But the Thali system won’t do that automatically.

If email were exchanged directly among peers, rather than through relays, then I might never have felt the need to bind identity to individual messages. Since email travels through relays, though, I would still like to assure you that email “from” me really is from me, as well as protecting it from the prying eyes of intermediaries and servers. But I find Yaron’s argument persuasive. The potential harm to me may outweigh the benefit to you.

[1] End-to-End isn’t, by the way, a Gmail-only thing. You can use it in any text entry field in the Chrome browser to compose a signed/encrypted message. I don’t use Gmail but was able to use End-to-End to send a protected message from That entails copying and pasting though, which presumably won’t be necessary if End-to-End is integrated into Gmail.

[2] Signing and encryption aren’t necessarily joined at the hip. Depending on how the technologies are implemented, it may be possible to sign without encrypting, encrypt without signing, or sign and encrypt. I tried the End-to-End extension and found that it doesn’t do bare signatures but does encrypt with or without signatures. So you could use it to protect messages without binding your identity to them.

Jeremy Dorn’s excellent JSON forms editor

The lingua franca for web data is a format called JSON (JavaScript Object Notation). It’s easier for people and machines to read and write than its predecessor, XML. And because JSON is JavaScript it’s a natural choice for a web increasingly powered by JavaScript code.

Like XML before it, though, JSON is only easy for some people to read and write: programmers. Most people need their interaction with JSON data to be mediated by HTML forms. As you’d expect, there are a bunch of JavaScript libraries that can do that mediation. Until recently, though, I hadn’t found one that felt right.

There is an embarrassment of riches. In any category of open source software where that’s true, evaluating the choices becomes an interesting challenge. Some useful criteria:

- Activity. Recent activity (commits, discussion, pull requests) by one or more contributors sends a strong positive signal.

- Demo. A live online demo, when feasible and if comprehensive, can help you decide whether to invest in exploring the project more deeply.

- Documentation. Clear, concise, and complete documentation also helps you decide whether to investigate more deeply.

- Size. Other things being equal, less code is more.

- Self-sufficiency. Other things being equal, less dependency on other code is more.

In my search for a JSON forms editor, these criteria led me to Jeremy Dorn’s json-editor. It’s small, self-contained, and wonderfully capable. And the demo is spectacular. Using it, I learned everything I needed to know about the tool, and verified that it met all my requirements. Because json-editor supports JSON schema, I was even able to prototype the data structures I was planning to use and then interact with them directly in the live demo.

Well done, Jeremy! You have not only created an excellent software tool. You have also exemplified how to present such a tool in a way that enables people to evaluate it quickly and thoroughly, then (if it meets the need) adopt it easily.

Everything is amazing and I am grateful

Last night I hung out with friends who hadn’t heard Louis CK’s profound rant, everything’s amazing and nobody’s happy. The central character is a guy experiencing WiFi on one of the first flights to offer it. He’s online at 30,000 feet, among the first ever to watch YouTube videos while hurtling across the sky. Then the service fails. “This is bullshit,” the guy gripes. Louis CK: “Really? The world owes you something you didn’t know existed 5 minutes ago?”

Today I’m on a flight from San Francisco to Boston. I just found out that a glitchy podcatcher failed to download the podcasts I was going to listen to for the next few hours. And, oh no! This isn’t a WiFi flight! The hipster response: “This sucks.” But I don’t feel that way. I’ll listen to those episodes of the Long Now Seminars and KUOW Speakers Forum some other time. (Perhaps, amazingly, streamed on demand to my phone while I’m out hiking with the dogs.) For now, instead, I am watching the blue dot on my phone’s map creep across Lake Tahoe. Which, if I incline my head to the right and look down, is visible directly below. And I am wishing my dad could be here to experience this miracle. He would have wept tears of joy.

Gene Udell was a Navy pilot, he flew the amphibious PBY Catalina, he understood the principles of flight and navigation. But he never took any of it for granted. Whenever we were in an airport, he’d look out at the jets on the concourse and say: “I know how it all works. But I still have a hard time believing those things can fly.”

It was the same when the Internet came along. He understood, roughly, how it worked. But it constantly amazed him. To have lived long enough to have such an experience was for him a privilege and an endless source of wonder and delight.

(What’s that town down there? Oh, Elko, Nevada.)

Was it always this way? I don’t think so. We learned how to make fire a long time ago but we’re still delighted to be able to do it today. Why do the newest and most advanced technologies provoke such ingratitude?

Maybe it’s because the newest and most advanced stuff builds on more layers of supporting technologies than we want to think about. Or maybe it’s that things evolve so quickly that we habituate and crave the next jolt of novelty. For one reason or another, our feeling of wonder gives way to a feeling of entitlement.

But dad, I’m here to tell you, it’s amazing. I wish you could be here sitting next to me watching that blue dot crawl across the map on my phone’s screen, identifying what we can see below. It would have made you so happy. And I promise you this: I will never take it for granted.

Can we tether email to “the truth”?

“I wish we had trackback for emails.” – Robert Scoble, circa 2006

My source for that quote is Jeff Sandquist, who hired both Robert Scoble and me to work at Microsoft. We are a company with a deeply-rooted email culture. Robert was bemoaning the lack of peripheral awareness that blogging culture had taught him to appreciate. In the blogosphere, as in later forms of social media, you are (mostly) guaranteed to discover responses to things you have written. In email culture there is no such guarantee, and that’s a bug.

I’m always on the lookout for ways to collaborate in shared spaces, and lately I’ve been getting good mileage out of Office 365 and OneDrive for Business. It’s a combination of hosted SharePoint, lightweight web apps, full-strength Windows apps, and sync among my various PCs, tablets, and phone. The pieces have come together in a way that reminds me of the excitement I felt when I first began experimenting with what we once called groupware.

But where email culture runs deep, it’s a challenge to build bridges between inboxes and shared spaces. Yesterday, for example, I sent an email with links to documents in my shared space on Office 365. One respondent advised me to use attachments rather than links. Another argued that links are superior because the reader is guaranteed to get the latest version. Both make valid points. You don’t want to hit a dead link if you’re reading email offline. But you don’t want to read a stale document if you’re online.

Why not do both? Use a link that resolves as an attachment when the email is sent, but retains its identity as a link. If the email is read offline, it functions as an attachment. But if the email is read online, it can function as a link too. Benefits include:

- If the document changed, the reader gets the current version

- If the document didn’t change, the reader knows it didn’t

- When the link resolves, the author sees a trackback

More broadly this idea reflects one of the core tenets of Thali. In distributed systems with copies of things floating around, there ought to be a canonical instance of each thing. “We call it the truth,” says Thali’s creator Yaron Goland. To the extent possible, we all need to be the source of truth for our own stuff, and we need to hold it as closely as we can.

Joint custody of data

Benjamin Mako Hill has long hosted his own email server. In Google Has Most Of My Email Because It Has All Of Yours, he rethinks that strategy after this conversation:

A few years ago, I was surprised to find out that my friend Peter Eckersley — a very privacy conscious person who is Technology Projects Director at the EFF — used Gmail. I asked him why he would willingly give Google copies of all his email. Peter pointed out that if all of your friends use Gmail, Google has your email anyway. Any time I email somebody who uses Gmail — and anytime they email me — Google has that email.”

Benjamin goes on to analyze his email archive and arrives at this sobering conclusion:

Despite the fact that I spend hundreds of dollars a year and hours of work to host my own email server, Google has about half of my personal email!

How could we manage our hosted lifebits in a way that enables our bits to commingle without loss of control? It’s easy in principle, though hard in practice. Here’s the easy-in-principle approach. An email is not a bag of bits that I send to you. It’s a bag of bits that I park in my own personal cloud, which is a cloud service that I trust, and/or a set of devices I own. I don’t send you the bits, I send you a link. Access to the bits, via that link, is governed by permissions I set.

You, conversely, authorize me to follow links that invite me to access your messages and replies. We both end up with archives of our conversational threads. Yes, of course, there’s nothing to prevent either of us from violating trust and sharing those threads with the world. But there’s no intermediary, we communicate directly, and we have joint custody of our mutual data in what Groove called a shared space.

There are, of course, all sorts of reasons why this is hard-in-practice and may never happen. But are they good reasons?