At its core, your browser is powered by an engine called the Document Object Model, hereafter DOM. You can think of the DOM as an outline, and the browser as an outline processor that shows and hides things, displays things in different ways, and even adds, removes, or rearranges things. Nowadays what you see, when you view a web page, is the result of a complex interaction between data and code. The data is the HTML content of the page, and the code is its JavaScript behavior. But these are slippery terms. A lot of content never originates as HTML, but is instead produced dynamically — by a web server, but also quite possibly in the browser as it manipulates the DOM. And a lot of behavior happens opportunistically in response to content on the page.
This arrangement has radical implications. For example, back in 2002 I invented LibraryLookup, a bookmarklet that noticed when you were visiting an Amazon or Barnes and Noble book page and offered a one-click search for that book in your local library. A few years later, a Firefox extension called Greasemonkey arrived on the scene. It offered two capabilities that, working together, enabled a zero-click LibraryLookup. First, it could call out to a web service. Second, it could modify the DOM based on the response. Putting these two things together, I wrote a script that would notice that you were visiting an Amazon book page, check to see if the book was available at your local library, and if so, insert a paragraph into the DOM that said: “Hey, it’s available at the [YOUR LIBRARY NAME] library!”
Is this kosher? I think so, but it’s a tricky question. At the time I made a short screencast that reflected on questions of ownership and fair use in an environment that’s designed and built to support intermediation and remixing. These questions were still largely hypothetical, though, because Firefox users who had also installed Greasemonkey were a very small number indeed.
But now, thanks to modern browser-independent JavaScript libraries like jQuery, those hypothetical questions are becoming very real. Here’s Phil Windley demonstrating his 2009 version of LibraryLookup:
The example comes from Phil’s recent essay The Forgotten Edge: Building a Purpose-Centric Web, which makes the case for contextualized browsing as enabled by libraries like jQuery and by infrastructure like that provided by Phil’s company, Kynetx.
In Phil’s next blog item, Claiming My Right to a Purpose-Centric Web: SideWiki, he asserts:
I claim the right to mash-up, remix, annotate, augment, and otherwise modify Web content for my purposes in my browser using any tool I choose and I extend to everyone else that same privilege.
That item grew a long tail of comments. It includes some interesting back-and-forth between Phil Windley and Dave Winer, but I want to focus on this observation from Greg Yardley:
Sites also generally come with a contract attached – some implicit (the view-through), some explicit (the click-through) – and these contracts, done correctly, are generally enforceable.
This whole post mystifies me, because you don’t have the the right to mash-up, remix, annotate, augment, and otherwise modify Web content – it’s not your content.
Earlier in the thread, Jeremy Pickens cited an example of such a contract: Google’s terms of service:
8.2 You should be aware that Content presented to you as part of the Services, including but not limited to advertisements in the Services and sponsored Content within the Services may be protected by intellectual property rights which are owned by the sponsors or advertisers who provide that Content to Google (or by other persons or companies on their behalf). You may not modify, rent, lease, loan, sell, distribute or create derivative works based on this Content (either in whole or in part) unless you have been specifically told that you may do so by Google or by the owners of that Content, in a separate agreement.
In response to Greg Yardley, Phil Rees cites fair use:
Actually we do have those rights.
I believe so too. Sooner or later, that belief will be tested.
After my March interview with Phil about Kynetx, I wrote:
There’s a continuum of ways in which I can modify a web page in a browser, ranging from font enlargement to translation to contexual overlays. I wouldn’t draw a line anywhere along that continuum. It seems to me that I’m entitled to view the world through any lens I choose.
This doesn’t only apply to my view of the virtual world, by the way. It will apply to my view of the physical world too. We don’t yet have magic glasses that overlay web prices on shelf items, or web reputations on store signage, but someday we will.
I can’t see how I could be prevented from creating a heads-up display — for realspace or cyberspace — that’s advantageous to me. But I’ve got a hunch that those magic glasses are going to be controversial.
I wonder if it’s going to boil down to magic glasses versus magic projectors. Or, in other words, private versus public augmentation of our experiences of the virtual and real worlds. I can wear my magic glasses, but I can’t necessarily project the view that I’m seeing.
I like the glasses vs. projectors analogy, and while the producers of content will probably fight even the glasses, I suspect they’ll lose in the end on the edge cases where it’s pretty clear that it’s glasses, and that the remixing is of content that you have rights to (whether through explicit license or fair use).
The hard part comes as things innovate: sometimes it’s unclear whether something is glasses or a projector. Not only is there no line between the various mods you can do to content (font, text-to-speech, overlays, remixes), there’s no line between browser and server (desktop and cloud) either. If I do remixes, Yahoo pipes style, where the work happens in the cloud, but for me only – is that glasses or projector?
And as innovators/entrepreneurs provide simpler and simpler services “glue” – and provide recipes and cookbooks for making your own glasses (Kyntext) – and potentially providing the processing capacity to do the refocusing – am I looking at their projector, or through my glasses?
I think they’re glasses, by the way.
“The hard part comes as things innovate: sometimes it’s unclear whether something is glasses or a projector.”
Exactly. And that’s where the metaphor will erode. I can lend you my real eyeglasses but when you look through them you won’t see my experience as rendered by the glasses.
But when the glasses are computational, I can show you exactly what they’re showing me. For example, I can invite you to look at the same computer screen I’m looking at.
Or, by means of screen-sharing technology, I can enable you to view that screen from a remote location.
We might then invite a third person to huddle with us around that virtual screen. Are we “publishing” yet? Most of us would say no.
But are we “publishing” if I make that shared view world-visible? Most of us would say yes.
Is there a way to draw a bright line between those two cases? I don’t see how.
Despite the fact that I argued against Windley the other day (he’s was a professor of mine many years ago, btw), I agree in principle that we do have those fair use rights. The problem I have is with the double standard. The same companies that claim they have fair use rights to “our” material (e.g. book scanning/search) argue from the other side of their mouth against our fair use rights when it comes to their material (e.g. our use of Magic Glasses on their Search Results).
At the very least, there needs to be consistency.
Furthermore, Gary is correct to point out the other complication: The line between glasses and projectors will continue to blur. For example, who says that you are not allowed to put on my glasses? Don’t I have the right to share my glasses with you?
And given that making a copy of my glasses is much easier in the digital realm than in the physical, there becomes little difference between sharing your glasses with a million people, and projecting the same end result to a million people.
And the more “sharing my glasses with a million people” starts to happen, the more a source like Google will be disintermediated. (Oh, and Bing, too :-) I do actually rotate my search engine usage and believe that any serious information professional should do the same; unfortunately most don’t.)
Google/Bing/Yahoo may start to die the same way newspapers are dying. They know it, and I think that’s why they’re fighting it, even if it involves a fair amount of double standard.
But when the glasses are computational, I can show you exactly what they’re showing me. For example, I can invite you to look at the same computer screen I’m looking at.
Ah, yes, exactly. We were composing our responses at the same time :-)
“Google/Bing/Yahoo may start to die the same way newspapers are dying.”
I don’t see how that follows. It’s just that there will be two complementary axes of personalization:
1. What those services can do for me, augmented by whatever knowledge of my friends and associates they already have, or that I choose to give them
2. What I can do for myself, in collaboration with my friends and associates.
“I do actually rotate my search engine usage”
I use multiple providers for every search, because I always want the benefit of a second opinion and a different perspective. For the same reason, I would seek maximum complementarity between search-driven and self-driven personalization.
But your point is well taken. It’s not just “serious information professionals” who should seek such diversity, it’s everyone. And that’s something we need to teach everyone, from an early age, as part of a thorough grounding in critical thinking.
There’s an MIT project which displays Amazon rating on shelf items.
http://www.ted.com/talks/pattie_maes_demos_the_sixth_sense.html
I love that Phil Windley looks up Howard Zinn. Very a propos. Perhaps he should look up Walter Benjamin, Art in the Age of Mechanical Reproduction, next.
Currently intellectual property law prioritizes the “original” creator over the remixer, except in some fairly narrowly defined cases. But the remixer is in a lot of cases the person adding the most value – by providing additional utility, by placing in a social or semantic context, etc., that the original creator didn’t anticipate.
As a lot of this content looks more like software than words-on-a-page, the free software model (writing it is free, distribution is ad hoc, making it useful gets you paid) seems to be increasingly appropriate.
I read this blog in Google reader, which remixes RSS feeds in a combined server/browser computational model. A paradigm of the proglassector. Who gets paid for this? It looks like google does, at least via the ads that appear at the bottom of many of the items, but at least in this case there’s a kind of “content VAT” that trickles down to the creator.
Re: “Google/Bing/Yahoo may start to die the same way newspapers are dying.” — I don’t see how that follows.
You’re right; I didn’t do a good job of justifying that leap. Let me try it this way:
If you can lend/give/license your glasses to me, who is to stop me from grinding part of your glasses down, and incorporating them into my glasses, so that I now have bi-focals? And if Gary then shares his glasses with me, and Phil Windley also licenses his to me, I can make tri-focals and even quad-focals. Multiply this by the hundreds of millions and pretty soon you have “P2P” information retrieval / search. Think Kazaa meets search.
Very few people will actually need to go visit the google.com (or bing.com) website anymore, because as long as the P2P “remixing”/”reusing”/”sharing” system has been seeded with Google-produced results those will propagate by themselves.
Every hour or two, you might want to refresh active query results by having a couple of nodes in the glasses-sharing P2P system go back to the Google and/or Bing source and see what the current results are.
But even with an occasional source refresh, look at what you’ve done; you’ve cut direct traffic to the search engine by orders of magnitude, because everyone is remixing/reusing the search engine results with each other.
And huge cuts in traffic translate to huge cuts in ad clicks. Therefore huge cuts in revenue.
Poof.. there goes the advertising model — and search engine revenue — up in smoke.
“Walter Benjamin, Art in the Age of Mechanical Reproduction”
What aspect of that essay seems relevant to you in this context?
“Think Kazaa meets search.”
Yes, that is what I’m thinking w/respect to glasses-sharing. But the laws of physics still apply. Until/unless latency falls much closer to zero, and we all cache and index the Net’s entire corpus on the devices in our pockets, central indexes will continue to play a key role.
Until/unless latency falls much closer to zero, and we all cache and index the Net’s entire corpus on the devices in our pockets, central indexes will continue to play a key role.
Back in the early 2000s when I used Kazaa (for strictly legal purposes, of course, ahem) I could easily get dozens of results for one of my queries in the 50s of milliseconds. Latency wasn’t an issue because the query was exponentially parallelized.
So I’m not talking about storing the index of the web in a distributed fashion, I’m talking about storing an index of Google results in a distributed fashion.
All you need to be able to do is get the top 3-4 results for your query back from the P2P network in reasonable time. The remaining results are below the fold, and can come a few seconds later with no perceptible difference to the user.
Through a combination of local caching and smart (topically similar) P2P neighborhood restructuring we can take care of lots of these issues. You could even create supernodes that acted as clearinghouses to speed up the vast majority (big head) of queries.
Furthermore, if this does not have the potential to work on the open web, why not restrict it to SMB company environments? Like some of the work that Barry Smyth is doing. Instead of a company hitting Google with the same queries over and over again, from different employees, a behind-the-firewall P2P result sharing network would (1) not only be faster than Google, but (2) provide more potential for the business to innovate on top of those results.
But look, even if P2P Search never overcomes those problems, and never succeeds against the big boys, the issue remains: Does Google allow reuse and remixture of their results in this manner? Does Bing?
If not, why not?
BTW, I’m an avid listener to your Interviews with Innovators series. Thank you!
(My wife and I especially liked the interviews with the biomass fuel fellow from the Northeast. Any chance of more interviews on that topic?)
“a behind-the-firewall P2P result sharing network”
Yes, that’s a huge opportunity to concentrate knowledge, spread awareness, and exploit serendipity. Not sure why P2P is relevant here though. If you’ve already got the results flowing through a chokepoint you can just leverage that directly.
“Does Google allow reuse and remixture of their results in this manner? Does Bing?”
Above my pay grade :-)
“My wife and I especially liked the interviews with the biomass fuel fellow from the Northeast.”
Jock Gill? Yeah, he’s a treasure. I absolutely want to do more of those, but don’t (yet) have as many connections into that realm.
Yes, Jock Gill. My wife got so excited that she immediately wanted to order a Polish wood gasification system. :-)
Myself, I want a soapstone fireplace. It captures a vast majority of the heat, then slowly radiates it out over the next 24 hours. Very efficient (and comfortable and beautiful): http://www.mountainflame.com/tulikivi_soapstone_fireplaces.htm
Re: P2P. Yes, from a technological standpoint, if you have a single chokepoint that would be the better thing to do. But remember, we also have to keep up the “glasses” vs. “projector” analogy. A single chokepoint is effectively a projector, and might no longer fall under fair use to remix Google results in this manner. But if you keep it P2P, you get to keep the “glasses” analogy, and therefore fair use.
Re: Walter Benjamin… Oh, I haven’t read that in 20 years, so this may be blurred. The ideas of glasses and projectors were prompted by a weighing of the relative roles of remixers and creators. One of Benjamin’s arguments was that mechanical reproduction would erode the “aura” of unique/original art – that feeling of uniqueness and specialness that packages the art, its provenance and history, the biography of the artist, etc. If I can buy a spot-on reproduction of Van Gogh’s sunflowers, what does it mean to see it in the museum? Given the price of Sunflowers and the people who still go to museums, I don’t think Benjamin’s prediction about art was right. If anything widespread reproduction has enhanced the aura of much original art.
But we can grab hold of the idea, even so. Content remixers are reproducers, I suppose, and maybe they erode the heroic aura of the original a bit. Which is why Windley might look him up, along with the People’s History of the US.
“Myself, I want a soapstone fireplace.”
Fine for an open plan house where you can rely on space heating. And it looks like that Tulikivi uses the clean/efficient 2-phase combustion that’s becoming standard for all modern wood-burning appliances.
For a house that’s divided into lots of small rooms, though, you really want central heat. Here’s my report on how things worked out with the Eko: http://blog.jonudell.net/2009/01/11/central-heating-with-a-wood-gasification-boiler/.
“But if you keep it P2P, you get to keep the “glasses” analogy, and therefore fair use.”
Ah. Right you are.
“If anything widespread reproduction has enhanced the aura of much original art.”
That’s true for art that originates in the analog realm. But now that you mention it, I guess that creative work originating in the digital realm often doesn’t entail that same kind of distinction between original and reproduction. Setting aside the aspect of live performance, what is meaningful about the original version of a literary or artistic work that was created digitally? Even for the creator, that work is experienced only as a rendering or reproduction.
“Here’s my report on how things worked out with the Eko:”
Oh that’s great! I missed that blogpost, originally. Thanks for the link.
Totally fascinating. Seriously.
Heya Jon — I’ve been thinking about this blog post for the past decade now. Chewing on it in the back of my mind. I think this issue of user control is one of the most important issues still facing the web. And one that the users have continued to lose this entire past decade.
Friend of mine recently had some interesting thoughts in this regard: https://medium.com/@dtunkelang/putting-users-back-in-control-a2432c2dbeaa
Hello again!
> The big information platforms would have to expose APIs and offer terms that make
> middleware companies financially viable.
Tall order, eh?
Howdy!
Incredibly tall order. Impossibly so, even.. at least without regulation.
But the ability to self-curate — i.e. the ability to have our own magic glasses — is even more important today than it was back in 2009, is it not?
I don’t think we had the language for it that we do now.. but at the risk of “retconning” the thought process behind magic glasses, claiming one’s right to a purpose centric web, Kynetx, and the general notion of private augmentation, I feel like what you / Windley / I / others were starting to feel at the time was a loss of empowerment or control over the ability to curate our own online experiences.
I may be wrong, but I think the impulses that led to a desire for magic glasses came from what Pariser in 2011 would call Filter Bubbles.. algorithms whose optimization functions were aligned enough with our own that they could get us to engage/click more, but that were not really what we would have chosen or curated for ourselves. I may be wrong, but I think you/we etc felt the filter bubbles scratching at the door, and didn’t want to let them in.
It only got worse as the decade progressed, with the 2016 election, Cambridge Analytica, massive growth in the markets selling our personal data so that the algorithmic curators can be even more accurate, etc.
But nothing is as accurate as our ability to curate and choose for ourselves.