Something about the title of this week’s Perspectives interview, OpenSearch federation with Search Server 2008, has been nagging me ever since I wrote it. In the interview, Richard Riley and Keller Smith describe how the new Microsoft search server can extend its reach by sending queries to other search services that can return results as OpenSearch-compliant RSS or Atom feeds.
We call this activity federation, but the enabling technology is syndication. So is the group of participating servers a federation, or is it a syndicate?
Some definitions of federation, from 1 dictionary.com and 2 Merriam-Webster:
1 a federated body formed by a number of nations, states, societies, unions, etc., each retaining control of its own internal affairs.
2 an encompassing political or societal entity formed by uniting smaller or more localized entities: as a: a federal government b: a union of organizations
That seems too formal, too heavyweight, for an OpenSearch-mediated search scenario. When you modify a search service to return results in the OpenSearch format, you’re not necessarily joining any kind of union. You’re just making it easier for other entities to latch onto your search results.
OpenSearch was announced on March 16, 2005, at the Web 2.0 conference. That same day I adapted my version of the InfoWorld search service to use it. There was nothing special about what I did, which is why it only took a few minutes. I just added a variant of the query URL that returned results as RSS, with a few minor extensions to comply with OpenSearch.
Then I registered my service with Amazon’s A9, searched A9 for “Jean Paoli”, and saw the combined results shown here.
This arguably was a federation, because you had to join the club in order to have results from your service show up in A9. But nothing about OpenSearch required things to work that way. Other services could consume my search feeds without requiring me to register with them, or permit them.
What’s more, any RSS reader could consume those feeds. Although I’d done the OpenSearch hack to showcase integration with A9, it turned out that I’d solved another problem without even intending to. It was now also possible for individuals to subscribe to InfoWorld queries.
OpenSearch can involve federation, but more fundamentally it’s about syndication. So, do the participating entities form a syndicate?
1 a: a group of persons or concerns who combine to carry out a particular transaction or project b: cartel c: a loose association of racketeers in control of organized crime
2 a group of individuals or organizations combined or making a joint effort to undertake some specific duty or carry out specific transactions or negotiations
That doesn’t seem right either. We can get closer by focusing on the definitions that emphasize simultaneous publication:
1 a business concern that sells materials for publication in a number of newspapers or periodicals simultaneously
2 to publish simultaneously, or supply for simultaneous publication, in a number of newspapers or other periodicals in different places: Her column is syndicated in 120 papers
But these definitions still involve more business coordination than OpenSearch, or feed syndication in general, require. If I use OpenSearch to publish a search service within the enterprise, I don’t need to make a formal agreement with the Search Server administrator in order to enable that server to include my search results. I just need to publish my results as an RSS feed, and tell that person I’ve done so. That same RSS feed is available to users who may wish to subscribe to searches performed directly on my service.
It’s the same on the open web. When you adopt a syndication-oriented architecture, small pieces can be loosely joined, or they can be more tightly coupled. But the underlying publish/subscribe mechanism doesn’t determine that choice.
Chewing on these definitions is more than a pedantic exercise for me. In my local community, I’m trying to show how a particular use of publish/subscribe technology — namely, calendar syndication — can solve an important problem for people, organizations, and the community as a whole.
Federation would clearly be the wrong word for the network of calendars that I’m trying to bring into existence. I’ve been using the word syndication instead. But now I suspect that’s the wrong word too. I want to convey that we can create small pieces, that they can be loosely joined, and that important network effects will emerge. I don’t yet know what word or phrase will make that cluster of concepts light up in people’s heads.
I’ve started using “re-presentation” in this sort of context, that is, in trying to combine the ideas of republishing and aggregation by the user.
It’s probably pushing it to say that network effect ideas are conjured up by images of representative democracy though, where big things (?!) happen because of the combined action of individual representatives…
I think I’d stick with federation. The dictionary is not a particular good source and doesn’t reflect practices around federated identity, federated search, etc.
In one sense, we have a bottom up federation arrangement that is achieved simply by offering and relying on coherent interfaces that allow federation as an interoperability arrangement.
So it is a different governance model, if you like, but I think federation (one out of many) is still the appropriate term.
I suppose you could call it aggregation but that would break down in the federated identity case, I think, even though that seems to me to be close to the search federation and integration case. (Does Open Search provide for notification of possible changes to previously aggregated results?)
Just pontificating here, but I think federation is a good generic term.
I’d stick with the term federated search whatever the underlying techology. Libraries have been using the phrse for a long time for something similar, but with a different underlying technology, z39.50, which is, as the wikipedia article on it says, a bit antiquated now.
Tony, you are thinking down the same path I am. As a application developer in the document imaging group at my employer we use the word “repurpose” a lot. Too often people do not decompose the information they want to identify the best (read as easiest, cheapest, etc) source. This is a stumbling block to easy acceptance of this concept in my eyes.
It feels natural to describe your community calendar project as building a library of sorts. Books exist independently and usefully outside of a library, but by collecting them in one place, and ensuring they all conform to a standard like Dewey Decimal, the set becomes much more useful than the sum of its parts.
We’ve been calling the process of combining such feeds aggregation/re-publishing, but in this light, perhaps it’s more apt to describe it as curation.
Approaching this from a different angle, I think you could describe it as being about creating a _one stop_ search engine or calendar.
There’s all this calendar information for your town out there, and what this technology providies is the ability to make a one-stop calendar where you can bring it together. That’s what RSS and syndication enables.
To take another example, RSS enables all the blogs I read to be brought together in a one-stop blog reader.
Just to summarise my last comment, I think the technology gives you the ability to create one-stop services.
(and btw, if you click the link from my name to my website in my previous comments, it takes you to a ‘A mega-site of Bible, Christian and religious information and studies.’ site. that is not my site :-). it seems if you have a mistype .blogpot.com, instead of blogspot.com, as I did, it takes you there).
> perhaps it’s more apt to describe it
> as curation
Yes. Collaborative and loosely-coupled curation. Problem is, people outside the geek world, who haven’t come to take that for granted, have no mental model for such a thing.
> There’s all this calendar information
> for your town out there, and what this
> technology provides is the ability to
> make a one-stop calendar where you can
> bring it together.
That’s step two. Step one is that, well, there really isn’t all that much calendar info out there, at least not in electronic form, and especially not in a syndicatable electronic form.
Seeding the combined calendar with lots of examples of what /could/ be provided, and then curated, is firstly a way to try to motivate the use of calendar-oriented software where currently what is mainly used, if anything, is wordprocessors and spreadsheets.
I was reading some other material today and distributed came up. I am not sure that works (distributed search and distributed identity sound nifty, but I think distributed search might have other nuances).
I am not keen on curation because of the heavy sense in which custodianship is part of that. I admit to checking a dictionary (well, Encarta’s), and discovered that curation is apparently a modern term or perhaps very specialized. It appears to be what curators do, and curators would appear to be control freaks (in a nice way, of course).
I should not have been surprised by return of the calendaring example, but I was. I thought that case was a form of aggregation with added requirement for coherence and the prospect of coordination. Maybe we are looking for a single term that would be seriously over-used if we charge it with all of the cases discussed here.
It really is syndication in the RSS sense, but I’m hoping to decouple that idea from all the baggage that blogging brings along. People have fixed views about blogging at this point, and outside of the geek tribe many of those views are not positive.
Among other things I’m hoping that if this idea of syndication can be conveyed in another context, one that’s more familiar, less threatening, and highly utilitarian, then we could eventually bring things back around to blog-style syndication as well.
“Pooling” maybe? I’ve been working on something similar for a couple of years now and called it OnePool (and since forever in closed alfa at onepool.com). The concept is basically Freeman & Gelernter’s Lifestream project from over a decade ago, using rss to feed it. You can create pools on the fly, using date-ranges, tags, saved searches, and all that stuff. I’m searching through my virtual, dynamic/elastic pools using a “pool search” of course :)
Best,
Chris.
Pooling is good in one way, being a familiar metaphor. But pools are supposed to stay put, not flow, and I really need to convey the idea of flow.
Here’s a nice example from May 10:
07:30 PM Animaterra Women’s Chorus Concert Benefits AVEO (aveo)
07:30 PM Animaterra Women’s Chorus (eventful: Keene Unitarian Universalist Church)
When I first started seeing these kinds of duplicates, my instinct was to suppress them. But actually now I think that’d be wrong. It’s cool to see multiple sources reporting the same event. It demonstrates buzz, and triangulates on the event from different points of view.
In this case, the first entry comes from the aveo.org calendar which is (as of a few days ago) exporting iCal.
The second comes from Eventful:
http://eventful.com/events/keene/animaterra-womens-chorus-/E0-001-010080339-7#box-details
And it isn’t one of the events I’ve posted there to seed this process. It’s organic. The fact that it says “Posted by evdb” tells me it came not from a registered individual user, but rather from an Eventful partner site. Namely:
http://www.newhampshire.com/calendar/details.aspx?eventID=10339&eventDate=05/10/2008
This is a great example of kinds of network effects I’m trying to illustrate. Animaterra promoted their event to NewHampshire.com, which syndicated to Eventful, and from there to elmcity.info, and from there to citizenkeene.ning.com and potentially elsewhere.
Meanwhile AVEO, as a beneficiary of the event, promoted it on its own website, and that info was also syndicated, in this case directly to elmcity.info (and again, from there potentially elsewhere).
I want people to appreciate how things flow in syndication networks, how information can circulate without loss of fidelity, and why that’s such a powerful way to spread a message.
The right word may help, and syndication may finally be the right word. But as with blogging, you can’t really convey the network effects to people until they put some skin in the game, and experience them happening w/respect to their own stuff. Same here I think.
This is off topic but you’ve not yet written the topic for this to be on, to wit, your conversation with Lucas Gonze: what is the itsy or bitsy service which provides a short unique identifier for the content of a file (if I understood correctly)?
I haven’t look tooo hard but I’ve only found stuff about spiders.
Cheers-
-Michael
> what is the itsy or bitsy service
bitzi
Barn-raising gets at the lack of formal membership and the collaborative aspect, maybe. But there’s still the flow to be considered.
Mosaic gets at the idea of small pieces forming something bigger.
‘Germination’ popped into mind (vs syndication, federation, etc)