A federated Wikipedia

Writing for the Chronicle of Higher Education in 2012, Timothy Messer-Kruse described his failed efforts to penetrate Wikipedia’s gravitational field. He begins:

For the past 10 years I’ve immersed myself in the details of one of the most famous events in American labor history, the Haymarket riot and trial of 1886. Along the way I’ve written two books and a couple of articles about the episode. In some circles that affords me a presumption of expertise on the subject. Not, however, on Wikipedia.

His tale of woe will be familiar to countless domain experts who thought Wikipedia was the encyclopedia anyone can edit but found otherwise. His research had led to the conclusion that a presumed fact, often repeated in the scholarly literature, was wrong. Saying so triggered a rejection based on Wikipedia’s policy on reliable sources and undue weight. Here was the ensuing exchange:

Explain to me, then, how a ‘minority’ source with facts on its side would ever appear against a wrong ‘majority’ one?” I asked the Wiki-gatekeeper. He responded, “You’re more than welcome to discuss reliable sources here, that’s what the talk page is for. However, you might want to have a quick look at Wikipedia’s civility policy.

(You can relive his adventure by visiting this revision of the article’s talk page and clicking the Next edit link a half-dozen times. You have to dig to find backstories like this one. But to Wikipedia’s credit, they are preserved and can be found.)

Timothy Messer-Kruse’s Wikipedia contributions page summarizes his brief career as a Wikipedia editor. He battled the gatekeepers for a short while, then sensibly retreated. As have others. In The Closed, Unfriendly World of Wikipedia, Internet search expert Danny Sullivan blogged his failed effort to offer some of his expertise. MIT Technology Review contributor Tom Simonite, in The Decine of Wikipedia, calls Wikipedia “a crushing bureaucracy with an often abrasive atmosphere that deters newcomers” and concludes:

Today’s Wikipedia, even with its middling quality and poor representation of the world’s diversity, could be the best encyclopedia we will get.

That would be a sad outcome. It may be avoidable, but only if we take seriously the last of Wikipedia’s Five pillars. “Wikipedia has no firm rules,” that foundational page says, it has “policies and guidelines, but they are not carved in stone.” Here is the policy that most desperately needs to change: Content forking:

A point of view (POV) fork is a content fork deliberately created to avoid neutral point of view guidelines, often to avoid or highlight negative or positive viewpoints or facts. All POV forks are undesirable on Wikipedia, as they avoid consensus building and therefore violate one of our most important policies.

That policy places Wikipedia on the wrong side of history. Not too long ago, we debated whether a distributed version control system (DVCS) could possibly work, and regarded forking an open source project as a catastrophe. Now GitHub is the center of an open source universe in which DVCS-supported forking is one of the gears of progress.

Meanwhile, as we near the 20th anniversary of wiki software, its inventor Ward Cunningham is busily reimagining his creation. I’ve written a lot lately about his new federated wiki, an implementation of the wiki idea that values a chorus of voices. In the federated wiki you fork pages of interest and may edit them. If you do, your changes may or may not be noticed. If they are noticed they may or may not be merged. But they belong to the network graph that grows around the page. They are discoverable.

In Federated Education: New Directions in Digital Collaboration, Mike Caulfield offers this key insight about federated wiki:

Wiki is a relentless consensus engine. That’s useful.

But here’s the thing. You want the consensus engine, eventually. But you don’t want it at first.

How can we ease the relentlessness of Wikipedia’s consensus engine? Here’s a telling comment posted to Timothy Messer-Kruse’s User talk page after his Chronicle essay appeared:

Great article. Next time just go ahead and make all of your changes in one edit, without hesitation. If you are reverted, then make a reasonable educated complaint in the talk page of the article (or simply write another article for the Chronicle, or a blog post). Other people with more, eh, “wikiexperience” will be able to look at your edit, review the changes, and make them stand.

To “write another article for the Chronicle, or a blog post” is, of course, a way of forking the Wikipedia article. So why not encourage that? There aren’t an infinite number of people in the world who have deep knowledge of the Haymarket affair and are inclined to share it. The network graph showing who forked that Wikipedia article, and made substantive contributions, needn’t be overwhelming. Timothy Messer-Kruse’s fork might or might not emerge as authoritative in the judgement of Wikipedia but also of the world. If it did, Wikipedia might or might not choose to merge it. But if the consensus engine is willing to listen for a while to a chorus of voices, it may be able to recruit and retain more of the voices it needs.

Thoughts in motion

In Federated Wiki for teaching and learning basic composition I speculated about a FedWiki plugin that would surface the version history of individual paragraphs within an essay. Over the weekend I prototyped that plugin and you can see it in action here. The paragraph that begins with “The relevance to SFW” is the one that I featured in the original blog post. On the wiki, in a forked copy of the Kate Bowles essay that prompted my inquiry, I’ve injected a plugin that lists paragraphs that have changed at least once since originally written, and that displays the version-to-version changes for each. On that page the plugin shows up in light yellow. If you click the + that precedes “The relevance to SFW” there, you should see the same record of changes I reported in the blog post. It looks like this:

This view improves on the mock-up shown in the original blog post, adding color-coded highlighting of version-to-version differences. I think it’s a striking illustration of how a thought evolves through a sequence of written snapshots. It reminds me of Michael Rubinstein’s astonishing TED talk on a video processing technique that reveals and amplifies normally undetectable color change and motion.

If Kate had been using a conventional writing tool, it would be much harder to observe what we see here. But in FedWiki a paragraph is special. It has its own version history. Every time you open up a paragraph to change it, that change is recorded. To the extent there’s a correspondence between paragraphs and thoughts — and in prose that is often the case — FedWiki intrinsically enables the study of those thoughts in motion.

When computers make visible what was formerly invisible there is always an initial rush of excitement. Then the question becomes: Is this useful? And if so, in what ways?

I can envision two primary uses of this technique. First, for writers. We all compose differently, and not everyone will want to be able to replay the changes to an individual paragraph. But if you do want to, conventional tools aren’t much help. In Google Docs, for example, you can roll through versions of a whole document but it’s not easy to focus on how a single paragraph changes.

A second use, as suggested in the original post, is for teachers of writing and their students. In other domains, such as music, computers have become powerful tools for studying finished compositions. Adrian Holovaty’s Soundslice, for example, makes YouTube performances accessible to study and annotation. In that case there’s nothing hidden, the tool’s job is to slow things down and provide a synchronized framework for annotation. But what if you wanted to study the process of musical composition? Then you’d want the composition to occur in an environment that records changes in chunks of varying granularity that correspond to the structure of the piece.

Because FedWiki naturally divides writing into paragraph chunks, it enables us to see how paragraph-sized thoughts evolve. But a FedWiki page is not only a sequence of paragraphs. Other kinds of objects can be injected into the page by plugin that manage, for example, data sets and small scripts written in domain-specific languages. These things have their own version histories too.

Most knowledge workers, with the notable exception of software developers, don’t yet use tools that take version control for granted. That will change. The artifacts produced by knowledge work are simply too valuable to be managed any other way. As versioning tools evolve for other disciplines, we have the opportunity to rethink what those tools can do. I hope we’ll do that with sensitivity to the natural granularity of the material. In many cases, whole documents aren’t the right chunks.

Even in software development, of course, we are still working with document-sized chunks. Compilers know that programs are made of modules and functions, but editing tools don’t track changes that way and as a result GitHub can’t directly show us the history of an individual function. That would be useful for the same reasons I’ve mentioned. It would help programmers reflect on their own work, and enable teachers to show students more about how code is written and how it evolves.

Tools for structured writing aren’t a new idea, of course. There are reasons why they haven’t caught on. But FedWiki reminds us that there are also reasons to hope they will.

How Federated Wiki neighborhoods grow and change

Federated Wiki sites form neighborhoods that change dynamically as you navigate FedWiki space. Sites that are within your current neighborhood are special in two ways: you can link to them by names alone (versus full URLs), and you can search them.

Here’s one neighborhood I can join.

A row of flags (icons) in the bottom right corner of the screen (1) indicates that there are five sites in this neighborhood: my own and four others. The number next to the search box in the bottom middle (2) says that 772 pages can be searched. That number is the sum of all the pages in the neighborhood.

From each site in the neighborhood, FedWiki retrieves a summary called the sitemap. It is a list of all the pages on the site. Each item in the list has the page’s title, date, and complete first paragraph (which might be very short or very long). FedWiki’s built-in search uses sitemaps which means that it only sees the titles and first paragraphs of the pages in your neighborhood.

Here are the sites in this neighborhood:

  1. forage.ward.fed.wiki.org
  2. jon.sf.fedwikihappening.net
  3. sites.fed.wiki.org
  4. video.fed.wiki.org
  5. ward.fed.wiki.org

You can find these names by hovering over the row of flags. If you are technical you might also want to observe them in a JavaScript debugger. In this picture, I used Control-J in Chrome to launch the debugger, then clicked into the Console tab, then typed the name of the JavaScript variable that represents the neighborhood: wiki.neighborhood.

Why are these five sites in my neighborhood? It’s obvious that my own site, jon.sf.fedwikihappening.net, belongs. And since I’ve navigated to a page on forage.ward.fed.wiki.org, it’s not suprising to find that site in my neighborhood too. But what about the other three? Why are they included?

The answer is that Ward’s page includes references to sites.fed.wiki.org, video.fed.wiki.org, and ward.fed.wiki.org. A FedWiki reference looks like a paragraph, but its blue tint signals that it’s special. Unlike a normal paragraph, which you inject into the page using the HTML or Markdown plugin, a reference is injected using the Reference plugin. It’s a dynamic element that displays the flag, the page name, and synopsis (first paragraph) of the referenced page. It also adds that page’s origin site to the neighborhood.

Two of the five sites in this example neighborhood — jon.sf.fedwikihappening.net and forage.ward.fed.wiki.org — got there directly by way of navigation. The other three got there indirectly by way of references.

To add a reference to one of your own pages, you click the + symbol to add a factory, drag the flag (or favicon) of a remote FedWiki page, and drop it onto the factory.

To illustrate, I’ll start with a scratch page that has a factory ready to accept a drop.

In a second browser tab, I’ll navigate to forage.ward.fed.wiki.org’s Ward Cunningham page, the one with the three references we saw above. Then I’ll drag that page’s favicon into the first browser tab and drop it onto the factory. Dragging between browser tabs may be unfamiliar to you. It was to me as well, actually. But it’s a thing.

The setup in this example is:

Tab 1: http://jon.sf.fedwikihappening.net/view/welcome-visitors/view/scratch

Tab 2: http://forage.ward.fed.wiki.org/view/ward-cunningham

Here is the result:

How many sites are in this neighborhood? When I did this experiment, I predicted either 2 or 5. It would be 2 if the neighborhood included only my site and the origin of the referenced page. It would be 5 if FedWiki included, in addition, sites referenced on the referenced page. Things aren’t transitive in that way, it turns out, so the answer is 2.

Except that it isn’t. It’s 3! Look at the row of flags in the bottom right corner. There are three of them: jon.sf.fedwikihappening.net, forage.ward.fed.wiki.org, and mysteriously, fedwikihappening.rodwell.me. That’s Paul Rodwell’s site. How did he get into this neighborhood?

This closeup of the journal will help explain the mystery. The page was forked 5 days ago.

We can view the source of the page to find out more.

And here’s the answer. Early in the life of my scratch page I forked Paul Rodwell’s scratch page from fedwikihappening.rodwell.me.

So we’ve now discovered a third way to grow your neighborhood. First by navigating to remote pages directly. Second by including references to remote pages. And third by forking remote pages.

FedWiki for collaborative analysis of data

A FedWiki page presents one or more wiki pages side by side. This arrangement is called the lineup. During interactive use of FedWiki the lineup grows rightward as you navigate the federation. But you can also compose a lineup by forming an URL that describes a purposeful arrangement of wiki pages. In Federated Wiki for teaching and learning basic composition I composed two lineups. The first compares two versions of a page on Kate Bowles’ FedWiki site. The second compares two versions of that page from two different sites: mine and Kate’s. With these two lineups I’m exploring the notion that FedWiki could be a writers’ studio in which students watch their own paragraphs evolve, and also overlay suggestions from teachers (or other students).

In that example the order of wiki pages in the lineup isn’t important. You can compare versions left-to-right or right-to-left. But here’s another example where left-to-right sequence matters:

Link: Favorite Broccoli Recipes

URL: http://jon.sf.fedwikihappening.net/view/italian-broccoli/view/broccoli-fried-with-sesame-and-raspberry/view/favorite-broccoli-recipes


The tables shown in these wiki pages are made by a data plugin that accumulates facts and performs calculations. FedWiki has explored a number of these data plugins. This one implements the little language that you can see in these views of the text that lives in those embedded plugins:

On the Italian Broccoli page:

5 (calories) per (garlic clove)
200 (calories) per (bunch of broccoli)
SUM Italian Broccoli (calories)

On the Broccoli Fried With Sesame and Raspberry page:

100 (calories) per (tbsp sesame seed oil)
34 (calories) per (100 grams broccoli)


3 (tbsp sesame seed oil)
SUM (calories)
1 (100 grams broccoli)
SUM Broccoli Fried With Sesame Oil (calories)

On the Favorite Broccoli Recipes page:

Italian Broccoli (calories)


Broccoli Fried With Sesame Oil (calories)

Other plugins implement variations on this little language, and it’s surprisingly easy to create new ones. What I’m especially drawing attention to here, though, is that the lineup of wiki pages forms a left-to-right pipeline. Facts and calculations flow not only downward within a wiki page, but also rightward through a pipeline of wiki pages.

And that pipeline, as we’ve seen, can be composed of pages from one site, or of pages drawn from several sites. I could provide one set of facts, you could provide an alternative set of facts, anyone could build a pipeline that evaluates both. It’s a beautiful way to enable the collaborative production and analysis of data.

Federated Wiki for teaching and learning basic composition

The FedWikiHappening has mainly explored Federated Wiki as an environment for collaborative writing. But the underlying software is rich with unexplored capability. It is, among many other possible uses, a great platform for the teaching and learning of basic writing skills.

Every page in FedWiki is backed by two data structures. The story is a sequence of paragraphs. The journal is a sequence of actions that add, edit, move, or delete paragraphs. Because editing is paragraph-oriented, the progressive rewriting of a paragraph is recorded in the journal.

I once taught an introductory writing class to undergraduates. Part of my method was to awaken students to the notion that paragraphs can and should evolve, and that it’s useful to observe and discuss that evolution. In FedWiki the evolution of a paragraph is not directly visible, but it’s available just below the surface. Here’s a beautiful example from a Kate Bowles essay called Sentences that get things done. The essay emerged in response to a collaborative riff that ends with Kate’s title. But here let’s watch one paragraph in Kate’s essay grow and change.

1. The relevance to wiki is that vernacular language is both capable of sustaining narrative and does not depend on citation.

2. The relevance to SFW is that vernacular language is both capable of sustaining narrative and does not depend on citation.

3. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation.

4. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed.

5. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed, and so becomes a practice of creating sentences that get things done, rather than further intensification of the spectacle of heroic individuals doing things.

6. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed, and so becomes a practice of collaboratively creating sentences that get things done, rather than further intensification of the spectacle of heroic individuals doing things.

7. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed, and so becomes a practice of collaboratively creating sentences that get things done, as a counterpoint to the intensification of heroic individuals doing things.

8. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed, and so becomes a practice of collaboratively creating sentences that get things done.

Version 4 might have been a keeper. But something propelled Kate to work through versions 5, 6, and 7. In the final version we see what she was reaching for: a way to land on the sentence that is both the essay’s title and a reference to the context from which the essay arose.

Any kind of web client software, running in the browser or in the cloud, could access that essay’s journal and surface that paragraph history. The FedWiki API (application programming interface) is simple and universal: just subtract /view from a FedWiki URL and append .json to the name of a page. So, for example:

Kate’s page: http://kate.au.fedwikihappening.net/view/sentences-that-get-things-done

The API for Kate’s page: http://kate.au.fedwikihappening.net/sentences-that-get-things-done.json

We can also construct URLs that arrange versions side by side. Here’s a FedWiki lineup that arranges two versions of the paragraph side by side and in context:


Now imagine that I’m the teacher, Kate is the student, I’ve forked Kate’s essay, and I’ve written version 8 as an example for Kate. Here’s an URL that arranges her version alongside mine:


I would love to help build tools that mine FedWiki’s latent ability to support the teaching and learning of prose composition. And I would equally love using those tools to facilitate that teaching and learning.

Individual voices in the Federated Wiki chorus

In recent days I’ve been immersed in the Federated Wiki Happening, a group exploration of Ward Cunningham’s Smallest Federated Wiki (SFW). When I first saw what Ward was up to, nearly a year ago, I resisted the temptation to dive in because I knew it would be a long and deep dive that I couldn’t make time for. But when Mike Caulfield brought together a diverse group of like-minded scholars for the #fedwikihappening, I had the time and took the plunge. It’s been a joyful experience that reminds me of two bygone eras. The first was the dawn of the web, when I built the BYTE website and explored the Internet’s precursors to today’s social software. The second was the dawn of the blogosphere, when I immersed myself in Radio UserLand and RSS.

During both of those eras I participated in online communities enaged in, among other things, the discovery of emergent uses of the networked software that enabled those communities to exist. The interplay of social and technological dynamics was exciting in ways I’d almost forgotten. This week, the FedWikiHappening took me there again.

I want to explain why but, as Mike says today, so much has happened so quickly that it’s hard to know where to begin. For now, I’ll choose a single narrative thread: identity.

SFW inverts the traditional wiki model, which enables many authors to work on a canonical page. In SFW there is no canonical page. We all create our own pages and edit them exclusively. But we can also copy pages from others, and make changes. Others may (or may not) notice those changes, and may (or may not) merge the changes.

In this respect SFW resembles GitHub, and its terminology — you “fork” a page from an “origin site” — invites the comparison. But SFW is looser than GitHub. What GitHub calls a pull request, for example, isn’t (yet) a well-developed feature of SFW. And while attribution is crystal-clear in GitHub — you always know who made a contribution — it is (by design) somewhat vague in SFW. In the Chorus of Voices that Ward envisions, individual voices are not easy to discern.

That notion was hard for some of us in The Happening, myself included, to swallow. In SFW we were represented not as avatars with pictures but as neutral “flags” made of color gradients. Identity was Discoverable But Not Obvious.

Then Alex North cracked the code. He read through the FedWiki sources, found the hook for uploading the favicon that serves as SFW’s flag/avatar, and worked out a procedure for using that hook to upload an image.

The next day I worked out a Windows-friendly variant of Alex’s method and uploaded my own image. Meanwhile a few other Happening participants used Alex’s method to replace their colored gradients with photos.

The next day Mike Caulfield bowed to the will of the people and uploaded a batch of photos on behalf of participants unable to cope with Alex’s admittedly geeky hack. Suddenly the Happening looked more like a normal social network, where everyone’s contributions have identifying photos.

That was a victory, but not an unqualified one.

It was a victory in part because Alex showed the group that SFW is web software, and like all web software is radically open to unintended uses. Also, of course, because we were able to alter the system in response to a perceived need.

And yet, we may have decided too quickly not to explore a mode of collaboration that favors the chorus over the individual voice. Can we work together effectively that way, in a federated system that ultimately gives us full control of our own data? That remains an open question for me, one of many that the Happening has prompted me to ask and explore.

How Thali could make the Smallest Federated Wiki even smaller

Thanks to my friend Mike Caulfield, an educational technologist who’s been digging into Ward Cunningham’s Smallest Federated Wiki, I’ve now got a much clearer idea of how SFW and Thali could play together and why they should.

Mike’s recent series on SFW is the best review and analysis of Ward’s newest creation that I’ve seen:




I had dipped a toe into the SFW water but there’s a learning curve and Mike climbed it before I could. Today he jumpstarted me by setting me up with a node of an SFW federation he’s hosting on AWS. Here I am participating in a wiki federation with some friends in the ed-tech tribe. We are able to do this because Mike provisioned SFW instances for each of us.

What’s the Thali connection? Well, in the first few seconds of http://screencast.com/t/fRlahVd0EK5 you see Mike provisioning a node in a federation he’s hosting on AWS. That’s the minimum bar for SFW: you need an instance of the server. Most people can’t or won’t leap over that bar.

But the server’s a pretty small piece of the pie. Most of SFW runs in the browser. There’s a lot there, and it’s well-architected for growth.

A server implementation for Thali would enable lots more people to create and participate in Wiki federations, by running SFW on their own devices and syncing opportunistically with peers on friends’ devices. Since the existing Sinatra-based SFW is CouchDB-aware, Thali — based on Couchbase Lite — should provide a comfortable home.

Why would people want to use SFW? Mike’s posts and screencasts point to a world in which GitHub-like collaboration breaks out of the geek ghetto and becomes a natural way for all kinds of teachers and learners to collaborate.

Ward points to that possibility and others in a series of SFW screencasts at http://vimeo.com/channels/wiki. I’d seen a few, tonight I went back and watched the rest. Some highlights:

On forking and comparing

An inline calculator plugin (in 25 lines of CoffeeScript!)

Visualization of in-page data

These demos really capture the idea of the universal canvas (http://www.infoworld.com/d/developer-world/we-need-universal-canvas-doesnt-suck-130) that I’ve dreamed of for a long time.

My 2006 InfoWorld article said, by the way,

Here’s the best definition of the universal canvas: ‘Most people would prefer a single, unified environment that adapts to whichever environment they are working in, moves transparently between local and remote services and applications, and is largely device-independent — a kind of universal canvas for the Internet Age.’

You might expect to find that definition in a Google white paper from 2006. Ironically, it comes from a Microsoft white paper from 2000, announcing a “Next Generation Internet” initiative called .NET.

You never know how things will turn out.