Faithful reanimation of born-digital artifacts

Here’s a glimpse of a capability I’ve long imagined:

It’s a picture of me browsing my December 1995 BYTE column, using the same browser — Netscape Navigator 4.0 — that I used when I wrote the column 20 years ago. You can try this yourself, for any archived website, using oldweb.today, a new project by the brilliant web archivist Ilya Kreymer. I’ve gotten to know Ilya by way of Hypothesis. We use his pywb proxy (hosted at via.hypothes.is) to inject our annotation software into web pages.

The rhizome.org blog has the scoop on oldweb.today’s lightweight emulation strategy. It’s a harbinger of things to come. Much of our cultural heritage — our words, our still and moving pictures, our sounds, our data — is born digital. Soon almost everything will be. It won’t be enough to archive our digital artifacts. We’ll also need to archive the software that accesses and renders them. And we’ll need systems that retrieve and animate that software so it, in turn, can retrieve and animate the data.

The enabling technologies are coming together quickly. Cloud-based computing keeps getting better and cheaper. The Docker revolution makes it radically easier to save and recreate complex software configurations. For the most part we benefit from this new infrastructure without noticing it or thinking about it, and that’s a good thing.

With oldweb.today, though, it’s hard not to notice or think about the infrastructure that delivers this magic into your browser. That’s a good thing too. It’s useful for everyone to have some awareness of what’s required to experience cultural artifacts with original fidelity, or to reliably reproduce scientific experiments. Ilya’s project isn’t only a great way to revisit the early web. It’s also a signpost on the road to these futures.

The downtown space station

An article I read years ago, and wish I could find again, claimed that America’s first enclosed shopping malls were designed to evoke the space stations of the mid-twentieth-century sci-fi imagination. I don’t know if that’s true but they do share a common design: the long central axis, with attached pods bathed in artificial light and recirculated air.

Until we moved to Santa Rosa, I rarely visited an enclosed mall. There wasn’t (and still isn’t) one in Keene, NH, which suited me just fine. I’ve always been allergic to the experience that malls try to create.

Here, though, the Santa Rosa Plaza (5) is a obstacle I ofen encounter when I walk between our home (1) in the West End and Luann’s studio (6) in the SofA district.

There is, thankfully, an alternative. The Prince Memorial Greenway runs along both sides of Santa Rosa Creek (4). It takes you around or under both of the structures that so heinously divide our part of downtown from the main part: the 101 freeway and the mall.

I use the creek trail as often as I can. The areas under the freeway are brightened by vast painted and tiled murals. Wildlife sightings include fish, egrets, black-crowned night-herons and, one time, a merlin. I see some of my neighbors walking or biking along, and others — the homeless ones — encamped along the banks.

When I’m headed downtown, though, the path of least resistance is (1, 2, 3, 5, 7). As you can see it leads through the mall. Although 101 looks formidable, you can cross under it at many points visible in the photo: 3rd Street, 4th, 5th, and 6th. But there’s no getting around the mall! I’d have to detour south to 1st or north to 7th to avoid it.

So I end up walking through the mall a lot, along what would be 4th street if it still existed. I try to enjoy it, and if I pretend that I’m walking through a space station I almost can. But this space station has landed in the wrong place. It bisects downtown even more egregiously than the mighty 101 freeway. What the hell were they thinking?

Parking in San Francisco

Although we are country mice we do venture down to the city now and again, as we did Monday evening to meet friends for dinner. It is super uncool to arrive at a downtown San Francisco destination in a private automobile. Ambulation, Uber, skateboard, bike, even Segway would be better. But for us there isn’t a faster way to get there and back during non-commute hours so we drive. And when there’s no traffic that mostly makes sense. Mostly, of course, doesn’t include that final variable: parking.

We got to the restaurant at 7PM, pulled into Garage 1, and met the villain of our piece:

Villain: We close at 9.

Jon: OK. How much for 2 hours?

V: $25.

J: No sale.

A hundred feet south we enter Garage 2 and meet the saviour of the day:

Saviour: We close at 9.

Jon: OK. How much for 2 hours?

S: $15.

J: [thought bubble: Sold!]

S: But if you’re going to the restaurant, just park right there. [gestures at the prime spot in front of the restaurant’s entrance, marked ‘For Deliveries Only, No Parking’]

J: Thanks! [thought bubble: $0 < $15 < $25]

Not having been born yesterday I park there, and check in with the restauranteur:

Jon: That guy says I can park in front but I figured I’d check with you.

Restauranteur: Who the hell is that guy to say who parks in front of my restaurant?

J: That’s why I’m checking. So, it is OK?

R: Sure.

J: Thanks! [thought bubble: Go figure!]

Passenger rail in the North Bay

In May 1906, Collier’s published Jack London’s The Story of an Eyewitness, his report on the devastation wrought by the San Francisco earthquake. Earle Labor’s recent biography of London explains how the writer was able to commute from his home in Glen Ellen, on the slope of Sonoma Mountain, to the burning city.

“EARTHQUAKE” is the opening entry in Charmian’s diary for Wednesday, April 18, 1906.

By six o’clock Jack and Charmian were on horseback, riding up the mountain to their new ranch. From that height they could plainly see the great columns of smoke rising over San Francisco and, to the north, Santa Rosa.

That afternoon they took the train to Santa Rosa. That smaller city has been hit as hard as San Francisco and was a mass of smoking rubble. From there they went to Oakland and took the ferry over to the inferno that San Francisco had become.

They took the train, in other words, from Santa Rosa where I live now, to Glen Ellen, to Sonoma, and finally to Oakland. That mode of transportation, once taken for granted, has been gone for decades.

Last week, walking to a coffee shop in Santa Rosa’s Railroad Square, I took this photo:

It’s the SMART (Sonoma-Marin Area Rail Transit) train on one of its first trial runs. I saw it again later in the week, as I was crawling down to San Francisco in a river of traffic on 101. There, in the marshlands of Novato, on a long-abandoned track, was the SMART train cruising along.

I know little about the politics of SMART, a flawed initiative that will not, anytime soon, restore the mobility that Jack and Charmian London enjoyed more than a hundred years ago. It’s a pale reflection of how BART planners in the 1950s imagined serving the North Bay:

When SMART is operational I’ll still need to drive down to San Francisco, or take the bus, and endure insane traffic delays if I have to travel during prime commute times. Sadly, the first phase of SMART won’t even reach the Larkspur ferry.

But one day next year Luann and I will be able to walk a few blocks to Railroad Square, get on a train, ride to Petaluma, enjoy its downtown for a few hours, and return home.

Jack and Charmian, if you’re reading this somehow, I know, it’s ridiculous, and I apologize. We really did screw things up, and this is a rather pathetic step the right direction. But at least it’s a start.

Facts of Flight

This page makes me smile:

It’s from a 1963 publication called Facts of Flight: Practical Information about Operation of Private Aircraft, which was available for 50 cents from the U.S Government Printing Office. Luann found it at a thrift shop the other day, and I can’t decide what’s most charming about it. The saturated colors remind me of the children’s books I began bringing home from our public library in that same year. The graphics, quaint at first glance, turn out to be stunningly effective. The explanation of the physics of flight, equally effective, is the same as the one I got from my father who had flown a PBY Catalina during World War Two.

This enjoyable and informative book is attributed to no author. It is simply “a manual prepared by the Flight Standards Service of the Federal Aviation Agency, to provide the private pilot with information essential to the safe operation of his aircraft.” I’m delighted to know that our government was once capable of writing an enjoyable and informative book. And I love the implicit message that any U.S. citizen could be an aviator in need of a pamphlet on atmospheric pressure, lift and drag, pitch, roll, and yaw, carburetor icing.

A new lease on life

Two weeks ago I underwent surgery to replace both of my hip joints. I’d been having trouble since the summer of 2012, when running became painful and I found that I couldn’t mount my bicycle by swinging my leg over the seat. These were signs of what would be diagnosed, in April 2015, as moderate-to-severe osteoarthritis. It could have been diagnosed two years earlier, when I presented symptoms I identified as a groin pull and pain in my quads. But I was still able to be very active then and, after a round of physical therapy, regained much of the range of motion I’d lost the year before. Deep down I knew something was really wrong but I convinced my doctor and physical therapist that it was all muscular, that I’d be able to work through it, and that there was no need for an x-ray.

In reality the disease was progressing, and although I returned to cycling, hiking, and running, things never felt right. There were other signs. During my final season of firewood stacking, in the summer of 2013, I found myself sitting on a bench pitching logs. Around the same time I tried working at a standing desk but quickly abandoned that experiment. Stretching didn’t help because it couldn’t. Cartilage was eroding, bone spurs were deforming the balls of my femurs.

After our move to California last year, the signs became impossible to ignore. Halfway through a 12-mile hike at Point Reyes I had to admit I was in real trouble. I went to see Luann’s physical therapist, he leveled with me, and I just about took the poor guy’s head off. Being active is incredibly important to me, I’ve always feared loss of mobility, and I catastrophized the situation.

Conventional wisdom has been that you want to postpone surgery as long as possible. That would mean years of increasing pain and loss of function. I visited the surgeon who would ultimately fix me, Briant Smith, and learned two vital facts. First, the hardware has improved so much that there’s less incentive to delay surgery in order to avoid a repeat. Second, the anterior direct method is massively superior to earlier methods.

Direct anterior hip replacement is a minimally invasive surgical technique. This approach involves a 3 to 4 inch incision on the front of the hip that allows the joint to be replaced by moving muscles aside along their natural tissue planes, without detaching any tendons. This approach often results in quicker recovery, less pain, and more normal function after hip replacement. Because the tendons aren’t detached from the hip during direct anterior hip replacement, hip precautions are typically not necessary. This allows patients to return to normal daily activities shortly after surgery with a reduced risk of dislocation.

At that point I didn’t know when this would happen. Years? Months? Dr. Smith said: “When you can’t take it anymore, we’ll fix it.” A month later I was in San Francisco for a week-long series of events. The first morning I walked two miles, a distance that had been well within my tolerance until then, and was incapacitated. I hobbled through the week, unable to sleep, and pinged Dr. Smith. “Sometimes the progression is linear,” he said, “and sometimes you fall off a cliff.”

On my next visit we discussed tactics. Most bilateral hip replacements are done serially, so you’ve got one relatively good leg to stand on during recovery. In my case, both joints were about equally bad. Should I replace both at once? My primary doc didn’t recommend that. And Dr. Smith didn’t exactly recommend it either. But he let me know that it’s doable. “It’s a big wallop,” he said, and a slower recovery, but people in my cohort — athletic, mid-to-late fifties (I’m 58) — are having successful outcomes, and it’s great to get it over with in one go.

So I signed up for the twofer, and then had a few months to wait, research the topic, and talk to veterans of the bilateral procedure. People were reporting almost unbelievably good outcomes. One fellow I met told me that his post-surgical pain was so minor that he’d needed nothing stronger than Tylenol to control it.

And that’s exactly how it went for me. An hour after surgery I walked from the hospital bed to a recliner, using a walker for safety more than for support. The next day I walked 500 feet, not because I was pushing myself but because the physical therapist invited me to and was confident that I could do it safely. The day after that I walked 1000 feet, navigated some steps in the PT gym to ensure I’d be OK entering our house, and went home.

It really was a big wallop, and the energy that’s gone into healing hasn’t left much for higher-order brain function. I’m sleeping a lot, and only gradually getting up to speed with my work. But I’m gaining mobility, strength, and flexibility every day. I used the walker for a week, then graduated to hiking poles — again, more for security than for support. And despite having major carpentry work done to both sides of my skeleton — hammers, saws, dremels — the pain has been minimal, and I too have needed nothing stronger than Tylenol. It’s reasonable to expect that I’ll be hiking and cycling in a month or so, and doing those things pain-free for the first time in years. My range of motion is already better along some axes than it was two weeks ago, and should continue to improve. There’s residual numbness in the skin over my quads, and some puffiness around the incisions, but these effects are normal and should resolve. (No staples or stitches, if you were wondering, they use superglue now.)

What accounts for this astonishingly good outcome? It seems to be a combination of factors. Hip repair has always been more straightforward than knee or shoulder repair, and with the anterior direct method it’s radically better than it was. That’s a recent development, my surgeon only began using the technique five years ago. But the physical therapist and occupational therapist in the hospital told me that they’ve only been seeing outcomes like mine for about two years.

What changed? There’s some fancy gear involved. The Hana table, for example, enables precise positioning, but it’s been around for longer than two years. Maturation of surgical technique is clearly a key factor. When I asked Dr. Smith about this, he said: “Speed matters.” It only took three hours to upgrade both of my hips. I was reminded of Atul Gawande’s 2007 New Yorker story The Checklist, a precursor to The Checklist Manifesto. Part of that story follows a team in Austria that had tried unsuccessfully to resuscitate people who drowned or were buried in avalanches, then managed to save one person, and soon afterward several more.

One step went right after another. And, because of the speed with which they did it, she had a chance.

Another key factor, I suspect, is that surgeons are seeing more patients like me: active people in their fifties who want to stay active, and who are realizing that surgery — this one in particular — need not be postponed.

It’s still not a walk in the park. I’m moving slowly, and there’s a long road ahead. But two weeks ago, every step I took made things worse. Now every step I take makes things better. It’s miraculous, and I could not be more grateful. For most of human history my condition meant a lifetime of diminishing mobility and increasing pain that medicine could do nothing to help. Within my lifetime a fix became possible. In just the last two years it’s gotten incredibly good. I’m well aware that there’s plenty that can’t be fixed at all, never mind fixed so well. I know I’m a lucky guy to be here in 2015 and, crucially, to have health insurance. But I also think Dr. Smith is a lucky guy. Being able to improve lives in the way he has improved mine, day in and day out, has to be an amazing joy and privilege.

A Labor Day meditation on the future of work

I’ve always felt weird about my interview with Jeff Bezos. Amazon’s PR people set it up as a follow-on to a keynote he gave, in 2006, about the suite of web services then coming online: cloud-based storage, computing, and human labor (i.e. Mechanical Turk). This was for InfoWorld, back when it was a magazine printed on paper. The PR folks were determined that we would put Bezos on the cover. We’d already written about all that stuff, so that wasn’t going to happen unless the keynote yielded something new and important.

Predictably it didn’t. But as they led me to the backstage room for the interview they kept pushing for the cover. And something must have snapped. Because when I pointed my camcorder at Amazon’s CEO and began talking, I became somebody I’d never been before and have never been since: the pushy reporter who is determined to frame a narrative, who steers the interview that way, and who interrupts all efforts on the subject’s part to answer questions not asked.

I’m not linking to the video because I still feel conflicted about it, but it’s on YouTube and I watched part of it recently in the wake of the New York Times story about Amazon’s workplace culture. Mechanical Turk was top of mind for me when I did that interview. I had been a customer of a Turk-based service called CastingWords, to which I had outsourced the transcription of several of my then-weekly podcasts. To prepare for the interview I signed up as a Turker and walked a mile in the shoes of a podcast transcriber. Here’s how I reported the experience at the time:

Podcasts are chopped up into six-minute segments. Last night, the segment I transcribed was worth $1.02, which will be credited to my Amazon.com account (and can then be transferred to a bank account) if my work passes several quality review checks. These checks are themselves implemented as HITs [human intelligence tasks].

My encounter with Turk Work wasn’t very satisfying. There was no context, no orderly progression, no sense of collaboration, no awareness of (or pride in) a finished product. If you’re running an MTurk-enabled business, you have to focus on throughput and efficiency. That drives you toward assembly-line tactics. But as Nathan McFarland noted in our conversation, you’d also like to reward excellence:

“We’d be willing to pay certain workers more and other workers less, but that’s not an option right now.”

So for now, that’s a hard constraint. Here’s another. From my perspective as a CastingWords customer, there are clearly some transcribers who have a better feel for my material than others. I don’t know who they are, though, or who’s transcribed which parts of various podcasts. If the goal is maximum throughput, that may be necessary. But that’s not my goal. Transcribing 620 minutes of audio for $260 in six days was darned impressive, but two months later I’m still only halfway through the final polishing, which I’m tackling in fits and starts. If this winds up being a long-term relationship, I’d rather identify the transcribers who do well on my stuff and give them whole tasks. And some transcribers would likely prefer that too. I sure would. Apart from the question of whether it’s in CastingWords’ interest to allow such relationships to form, the current MTurk architecture precludes the possibility.

I pressed Bezos hard on that point. Too hard, some thought at the time. Perhaps they were right. Perhaps it was none of my business to suggest that Mechanical Turk ought to optimize for worker satisfaction as well as for throughput.

My first encounter with a digital sweatshop was in 1993, in Bombay (as it was called then). I was visiting a software industrial park. In one company’s office I saw a room full of desks at which workers were digitizing maps of US telecommunication infrastructure. It seemed like dehumanizing work. Then I looked out the window at laborers carrying piles of bricks on their heads in 100-degree-Fahrenheit heat and realized that those map digitizers, in their air-conditioned office, had some of the best jobs in town.

I have no inside knowledge of Amazon’s blue-collar or white-collar workplace cultures, so I’m in no position to judge. Given that MTurk was a forerunner of the gig economy, though, here’s the follow-up question I wish I’d asked Jeff Bezos then and would still like to ask now. Do you believe that optimizing for customer satisfaction and worker satisfaction is a zero-sum game?

“It is always quiet enough to talk”

Our favorite local spot in Santa Rosa so far is The Toad in the Hole. It’s an English-style pub on the west end of downtown, an easy walk from our house. We like their brews, we like their Cornish pasty, and most of all we like their atmosphere. Sports bars aren’t really our thing, nor is soccer, but while soccer games are often playing on TV, they’re not intrusive and not usually a focus of attention.

Toad in the Hole wants to be a place where people come together to talk. The flip side of the menu reprints a 1946 George Orwell essay, The Moon Under Water, a fictionalized amalgam of the best features of all the pubs he knew and loved. At the center of the essay is this pivotal sentence:

“In the Moon Under Water it is always quiet enough to talk.”

I’m not any kind of audiophile but I’ve long been bothered by the awful acoustics in so many establishments. It’s hard to have a conversation in a room full of music and voices bouncing off hard surfaces. And while it’s easy to mitigate those sounds with baffles, that’s rarely done. One reason Toad in the Hole lives up to Orwell’s ideal is that it gets that right. It eluded me on our first few visits, but the other night I looked up and saw this:

You can see two acoustic baffles hangling from the ceiling. There are more like that all around the perimeter. It’s a subtle thing that most patrons may not notice. But it makes a huge difference!

It’s also a nice lesson in product design. Vanish into the woodwork, solve problems that people may not even recognize in a way they may not even notice, and let them just get on with what they care about.

Farewell to bookmarklets

The Hypothesis web annotation system is a browser-based application that talks to an annotation server, finds annotations for the current page (if any), highlights the selections to which the annotations refer, and dispays them in a sidebar alongside the page. We load a fair bit of JavaScript to do our job, and in the olden days we would simply have relied on bookmarklets to activate that code.

But times change, and bookmarklets are mostly dead. They’re not all dead yet. I expect I’ll still be able to use simple ones, like pinboard.in‘s, for a long time to come. But a new security regime limits the scope of what bookmarklets can do, and is forcing developers to create browser extensions instead — extensions that must be created and deployed differently for each browser.

So the realm of web applications that extend the browser is now becoming as balkanized as the realm of mobile apps. Developers target the dominant platform first — currently, that’s Chrome — and then port their extensions to other browsers as and when they can.

There’s another way to add behavior to a web page, though. You can send the page through an intermediate server (a proxy) that injects that behavior as the bookmarklet or extension would do. The Hypothesis proxy is via.hypothes.is, and you can use it to activate annotation for any web page — including this one — like so:

https://via.hypothes.is/blog.jonudell.net/2015/05/13/farewell-to-bookmarklets

The nice thing about this combo URL is that you can share the annotated page along with its annotation layer. Of course there are tradeoffs:

– Properly rewriting all the URLs in the proxied page is a tricky business.

– You can’t annotate pages behind paywalls (as you can with our bookmarklet and Chrome extension).

– You still need a way to activate the annotation layer for the current page. (You can use a bookmarklet to redirect to the proxy, but … sigh.)

The upshot is that we’re struggling to simplify the decision tree that Hypothesis users must traverse. You want to activate annotation for your current page? If in Chrome, install the extension, otherwise for Firefox or Safari or IE use the bookmarklet, unless the page enforces Content Security Policy, in which case use the proxy, unless the page is behind a paywall, in which case, sorry, you’re out of luck.

And that’s just for the desktop! Mobile browsers make things even crazier. I’m able to perform the gymnastics required to use the Hypothesis bookmarklet on my Windows Phone, on my iPhone, and (most acrobatically) on my Android tablet, and I really need to document those procedures, but it’s so depressing I’ve avoided doing so.

What seems most promising, for mobile browsers, is to add Hypothesis as an option that shows up when you ask to Share a web page. Activating annotation for the current page isn’t really Sharing, but the Share mechanism looks like the sanest way to redirect through the proxy, which in turn looks like the sanest way to activate the annotation layer in mobile browsers.

It looks reasonably straightforward to create an Android app that shows up when you Share a page and broadcasts the intent to view the proxied version of that page. Things look a bit more complex on iOS. Should there be a Safari-specific extension that redirects to the proxy? A system-wide Share extension that Chrome could also use? Both?

Of course activating Hypothesis on the mobile browser is only half the battle. As a sidebar-based app we manage pretty well on the desktop and large tablets, but are uncomfortable on smaller screens. If we’re routing through a proxy anyway, maybe the proxy should also do a Readability-style rewrite to simplify the page. We’re good at fuzzy anchoring and can anchor annotations to different representations of the “same” document.

There’s lots to do, and while bookmarklets will continue to help us do it, their era is over. Browsers are more standards-compliant than ever, which makes our mission to bring an annotation layer to the web easier than it would have been five or ten years ago. But browsers are now also less easily extensible and that’s a new challenge.

My last run

Here’s a picture of my last run.

I don’t mean last as in most recent. I mean last in my lifetime. What I thought was a groin pull a few years ago turns out to have been hip osteoarthritis.

So that sucks. It isn’t life-threatening. I am OK and will be better. It’s not breast cancer, or macular degeneration, or any number of worse things that friends and family have suffered through. I get that. But it’s going to be a hell of an adjustment for me. I’m an active person. I was always planning to be that guy who does mini-triathlons into his 80s. Turns out that wasn’t in the cards.

There’s a tendency to blame yourself. If only I hadn’t been a runner all those years. Or maybe if I hadn’t been a gymnast before that, and landed hard a bunch of times. But the orthopedic surgeon who will repair me, at some point, said no. It’s not your fault. Plenty of people run in their 80s. Some gene now being expressed says you’re not one of them.

I guess the strangest part of the experience is that I never located the problem in my hip. Or, actually, in both of them. Hip osteoarthritis can manifest as groin pain, so that’s what I told myself was happening — and only on the right side. Yeah, it shouldn’t go on for 3 years. Yeah, your quads shouldn’t hurt like hell after a run or a long hike. Yeah, you shouldn’t dramatically lose range of motion in your legs. Yeah, it shouldn’t keep you up at night.

But the brain is powerful. It enabled me to believe, for several years, that this was just one of those soft-tissue injuries I’ve worked through in the past. With stretching, and the right kind of exercise, I’d get through it, just like I always have.

Then I went to see Luann’s physical therapist and he leveled with me. This was 99% certain to be osteoarthritis. There was no other explanation for my symptoms. I feel bad for the PT because I went kinda ballistic on him. I knew he was right, and I started cursing like a sailor because I knew what it meant.

And suddenly things felt different. I began to visualize what was going on in my joints, and my subjective experience began to shift. A week later the orthopedic surgeon showed me the X-rays and that really clinched it. The balls in both of my ball-and-socket joints are deformed, it’ll get worse until I cry uncle and replace them. I’m feeling that now in a way I was denying for years.

The good news is that we live in a time when that’s possible. And that hips are way easier than knees. (Luann already replaced one, may face another.) And that we have health insurance. And that we’ve moved to a place where cycling — my best exercise at this point — is a year-round activity.

I don’t know if I’ll pull the trigger in a year, or two, or five. I’m guessing not longer than that. I took a 2-hour hike yesterday and I’m really feeling it today. That’ll only get worse. When they fix me I should be fine doing things like that.

But no more running, ever. I’ll adjust. But it’s hard to let go. And it’s hard to explain to Tuck. We brought him home in 2009, he was our first dog ever, and he got the job because he was the best of his siblings at running alongside me. Now he looks at me and wonders why we don’t do that anymore. Sorry, pal. Really wish we could.

Adventures in annotation

I just wrote my first blog post for Hypothesis, the web annotation startup I joined recently. In the post I talk about how a specific feature of the annotator — its ability to sync annotations across local and/or web-based copies of the same file — illustrates a general approach to working with copies of resources that may live in many places and answer to many names.

When I finished drafting the post I pinged Dan Whaley, founder of Hypothesis, to review it. Here’s the IRC chat transcript:

Jon: https://hypothes.is/?p=3705&preview=true

Dan: I'm annotating!

Jon: The preview URL?

Dan: :-)

I was a bit surprised. The preview URL was password-protected but annotations against it would not be, they’d show up in the public annotation stream. But hey, I’m all about transparency when appropriate, so bring it!

Over the next few minutes we traded annotations and I tweaked the post. Here’s a picture of Dan asking to add space around an element.

And then jgmac1106 jumped in.

That’s Greg McVerry, an enthusiastic user of Hypothesis. I’d been in touch with him earlier that day because he’d asked a question about local annotation of PDFs, we’d conversed, and I wrote the post partly to answer the question as broadly as possible. I couldn’t easily grant him access to the preview, but I’d sent him a copy of the post as an attachment. And suddenly there he was, contributing to the collaborative edit that Dan and I were doing. It was a nice surprise!

After I published the post I got another nice surprise. I had realized that the annotations on the preview would remain visible in Hypothesis. But when I cited it in an internal forum, Dan responded with the canonical WordPress URL, https://hypothes.is/blog/synchronizing-annotations-between-local-and-remote-pdfs/, and when I loaded that into a tab where Hypothesis was active, all the preview annotations were intact.

It took me a minute to realize how that was possible. A WordPress preview URL knows the eventual URL at which a post will appear, and encodes it in the HEAD section of the HTML document like so:

<link rel=”canonical” href=”https://hypothes.is/blog/synchronizing-annotations-between-local-and-remote-pdfs/”&gt;

When the Hypothesis service receives an annotation for the preview URL that declares a canonical URL, it remembers both as aliases of one another. That is, of course, exactly the point I was making in the post.

We hadn’t planned on this but, as a result, you can see the chatter that preceded publication of the post, as well as chatter since, through the lens of Hypothesis, at either the preview URL or the published URL.

Note that you don’t need to install the Hypothesis extension, or use the bookmarklet, to load Hypothesis on our blog, because it’s already embedded there. You only need to activate Hypothesis as shown here (click to play the mini-screencast).

I haven’t thought through all the collaborative possibilites this will enable, but it sure makes my spidey sense tingle.

Annotating the web: my new job

I’m delighted to announce that I’ll start a new job tomorrow, as product manager for Hypothesis. We’re a small team chartered to build open annotation software for the web. We expect it will be useful in many domains, including education, science, journalism, and government.

Some of you who know me in one or another of those contexts will be hearing from me in coming weeks. I’ll want to know how you annotate the web now, what you need annotation software to be, and how we can work together to meet that need.

I’ve long imagined a standards-based annotation layer for the web. Now’s my chance to help make it real. Exciting! If you’d like to get involved, ping judell at hypothes.is.

Can Santa Rosa become a city that thinks like the web?

I want cities to connect the dots between open government, web literacy, and citizen engagement. And I want to help the city I live in do that. For many years that city was Keene, NH, and while I can point to a few successes I can’t say that Keene is now a great model of the kinds of web-mediated engagement I envision.

Now that I live in Santa Rosa, I’ll try to help my new city connect those dots. The timing is fortuitous. In the wake of the 2013 Andy Lopez shooting Santa Rosa’s then-mayor Scott Bartley appointed an Open Government Task Force. The charter was:

…to develop a report that will inform the community about the current status of open and transparent government practice in the City of Santa Rosa; to review the exceptions, limitations and restrictions imposed by State or Federal law or Constitution; and to present options for improvement or additional best practices that the Mayor and City Council may wish to consider.

The final report, happily, takes a holistic view that encompasses not only government transparency but also citizen engagement. That’s the area in which I think I can help.

In The Church of One Tree: A civic parable I looked at how government and citizens interacted on a couple of issues, and explored some ways in which better web literacy could have improved those interactions.

Today the Press Democrat ran my editorial, Open government is a two-way street, which distills the pitch down to 600 words.

Can Santa Rosa become a city that thinks like the web? It seems genuinely to want that. If so, I’m here to help!

Remembering Bob Stout

A couple of years ago I wrote about recovering from an injury. At the time I thought it started as a pulled muscle that cascaded when my determination to keep running and cycling led me to compensate with various misalignments. The physical therapists I saw at the time agreed, so when the acute phase died down we worked on restoring the range of motion I’d lost in my right leg. I recovered some, but not all, of that range, and eased back into hiking, and then running and cycling.

Things haven’t felt right since, though. I’ve never regained my full range of motion in that leg, and I’ve been feeling a lot of discomfort in both quads as well as in the right groin where the initial problem began.

When I talk about full range of motion I mean something different from what most people mean. Take a look at this young man doing front stalders on the horizontal bar. Think about the leg and hip flexibility required to do that. In high school and in college I used to do stalders. Since then I never lived near a gym where I could swing on a high bar, though it’s something I still dream about, could do, and would do if there were a gym that had the gear and would let me use it.

It’s natural to assume that only a young person can be that flexible. Not so. There have been a few role models in my life who have shown me what’s physically possible in later life, and one of them was Bob Stout. In 1952 he was a U.S. Olympic gymnast. In 1972, when I was getting into the sport as a 15-year-old, he worked out with my high-school team and showed us that a man then in his late forties could still hold an iron cross on rings (something I could never do), swing high bar, and demonstrate the full hip extension required for the stalder. As far as I know he was still doing those things when he died at age 56, of a heart attack, while jogging.

I am 58. Until recently I could also reach nearly that same hip extension. Then, after the incident a couple of years ago, my flexibility diminished by a lot. The hypothesis I hope will prove true is that I got myself into a vicious cycle. An injury made it hard to stretch, so I avoided stretching, which made stretching even harder. Ditto for squatting. I’ve been avoiding it because it was uncomfortable, that made it even more uncomfortable.

The hypothesis I am afraid will prove true was suggested by a physical therapist I saw last week. He thinks I have osteoarthritis of the hip. If he’s right, I’m looking at no more running, reduced hiking, lots of pain management in order to maintain activities like hiking and cycling, and joint replacement at some point. I guess the X-ray will tell the tale, and that won’t be for a few weeks. Meanwhile, it’s suddenly a high priority to maximize strength and flexibility. Whatever the state of my hip joint(s) turns out to be, those will be key assets.

I know how to maintain those assets. I don’t know what the limits of recovery are, once muscle has been allowed to atrophy and connective tissue to tighten. So, I’m doing that experiment now. I’ve seen noticeable improvements just in the past few days. I have no idea how far I’ll get, but I’m grateful to Bob Stout for his inspiring example.

On getting paid (or not) to write

Danielle Lee writes the Urban Scientist blog for Scientific American. In 2013 she wrote a post about an ugly incident in which she was invited to write for Biology Online, asked about payment, declined the offer when she learned there would be none, and was called a whore. Scientific American took down the post an hour after it was published, late on a Friday, and then restored it the following Monday after verifying DNLee’s claim. The Biology Online editor’s name-calling was horrific. But his email also included a lesser insult:


You will enjoy a great deal of exposure from our 1.6 million monthly visitors.

I heard the same thing from Wired a few years ago, when I was invited to write an online column. They weren’t paying. But I was otherwise employed, I missed writing for a wider audience than my blog attracts, and I bought the claim that appearances on Wired.com’s home page would yield useful visibility, feedback, and engagement.

It didn’t work out that way. That was the loneliest writing gig I’ve ever had. On-site comments were few and far between. And reaction elsewhere — on Twitter, in the blogosphere — was anemic as well. I felt like I was talking to myself in an empty room while, below in the engine room, machines were talking to other machines, spinning page counters unconnected to any real audience.

Now I’m writing for InfoWorld again, and it’s a much more pleasant experience. That’s partly because InfoWorld does pay contributors. It’s not a living. The world has changed since I left InfoWorld 8 years ago, when there was still a print magazine. But there’s a real exchange of value.

As important to me, if not more important, is the connection to an audience. With InfoWorld I’m feeling that connection again. I can see conversations forming around my columns and features, and those conversations lead me to new ideas. That’s a dynamic I cherish.

Could I still earn a fulltime living writing about technology? Probably not, and I’m sometimes wistful about that. It’s something I do really well; arguably it’s my strongest talent. But the world’s supply of creative talent far exceeds the commercial demand for it. The vast majority of writers, artists, and musicians need day jobs. That can be a good thing. In my writerly niche it certainly is.

The kind of technology journalism I’ve practiced didn’t exist until BYTE came along in the 70s. Like all the tech publications that followed, we faced a dilemma. Do you hire practitioners who can learn to write? Or do you hire journalists who can learn about technology?

The best answer, of course, was (and is): Hire practitioners who are also reporters and writers. Then make sure they keep developing their practical knowledge and skills. At BYTE, and then at InfoWorld the first time around, I was blessed with the opportunity to do that. Such opportunity may not come again. That’s OK. I need to be able to deliver value in other ways too. Doing so will keep me from devolving into a pundit.

What’s not OK is writing for commercial publications that don’t pay. If there’s no market for something I want to write, I’ll put it here instead of on Medium or Facebook or some other site that earns in the currency of dollars but pays in the currency of (presumptive) attention.

Fringe benefits of the attention economy

A decade ago I captured the peak of my Internet fame in this screenshot:

That was near the end of an anomalous few years during which the top result, when searching Google for “jon,” was my blog. I enjoyed it while it lasted, knowing that Jon Stewart would inevitably eclipse me, as he did in 2005 and as many others have since.

I was among the first to write professionally for the web, so for a while many of the pages in Google’s index containing “jon” were mine. That was just a lucky first-mover advantage. I knew it would erode over time as, appropriately, it has.

I still enjoy some residual benefits, though. My blog’s popularity translated to Twitter when it appeared on the scene, and although my Twitter reach has grown only modestly since, I was recently reminded that it remains another kind of first-mover advantage.

When United charged me for an error it made on a recent flight reservation, none of the regular customer service channels were responsive. So:

And then:

I’m hardly a celebrity on Twitter, but airing my complaint in a way that 5000 people might notice got results. I was grateful, direct-messaged my thanks, and received this DM reply:


You’re welcome, be sure to tweet us if you need anything. ^HN

I certainly will. But your mileage will almost certainly vary. Of those who try this method, how many will have enough Twitter reach for United to worry about? When I mentioned that on Facebook, Tony Byrne said:


I’ve wondered a lot about the equity of this. Supposedly social customer supt costs 8x traditional cust supt but brands do it for precisely the reason you cite and they are very aware of your Klout score when you complain. Too many of us digerati remain too smug about this, as if we deserve special treatment for being active on Twitter…

There’s nothing new about the attention economy. But there are always new ways to unfairly distribute attention.

Online scientific collaboration: the sequel

In 2000 I was commissioned to write a report called Internet Groupware for Scientific Collaboration. That was before modern social media, before blogs even really got going. But arxiv.org was already well-established, and wikis and calendar services and Dave Winer’s proto-blog, Manila, and many kinds of discussion forums were relevant to my theme. On the standards front, RSS, MathML, and SVG were emerging. One of my premonitions, that lightweight and loosely-coupled web services would matter, turned out to be accurate. Another, the notion of a universal canvas for creating and editing words, pictures, data, and computation, remains part of the unevenly distributed future, though projects like IPython Notebook and Federated Wiki rekindle my hope that we’ll get there.

Now I’m writing an update to that report. There’s unfinished business to reconsider, but also much new activity. Scientific collaboration happens in social media and on blogs, obviously. It happens in scientific social media. It happens in and around open access journals. It happens on GitHub where you can find open software and open data projects in many scientific disciplines. It happens on Reddit, on StackExchange-based Q&A sites, on citizen science websites, and in other places I don’t even know about.

I want to interview researchers engaged in various aspects of online scientific collaboration. I’m well connected to some of the tribes I need to reach, but need to cast a wider net. I want to hear, from practitioners in natural sciences, social sciences, and digital humanities, about ways you and your colleagues, in disciplines near and far, do, and/or don’t, collaborate online, both in specific contexts (OA journals, academic social networks) and wider contexts (blogs, mainstream social media). How does your activity in those settings advance your work (or not)? How does it help connect your work to society at large.(or not)?

If you’re somebody who ought to be involved in this project, please do get in touch here or here. And if you know someone who ought to be involved, please pass this along.

Thanks!

Another Internet miracle!

I’m among the many fans of the entertaining physics lectures that made Walter Lewin a star of stage (MIT OpenCourseWare) and screen (YouTube). And I was among those saddened, last month, to learn that charges of harassment had ended his career on the OpenCourseWare stage.

When it severed its ties to Lewin, MIT made the controversial decision to remove his lectures from ocw.mit.edu. Searching for perspective on that decision, I landed on Scott Aronson’s blog where I found much useful discussion. One comment in particular, from Temi Remmen, had the ring of truth:

I agree Walter Lewin’s lectures should be made available through a different source so everyone around the world may enjoy them. Having known him for most of my life, I am not in the least surprised that this happened to him. None of us enjoy his downfall. However, he managed to alienate many of his peers, colleagues and people in his personal life to an extreme. It is my gut feeling, that prominent people at MIT had enough of his antics, in spite of his success as a teacher and brilliance as a scientist. In the scientific community, he is widely known for being very demeaning and insulting to those he does not feel are as intelligent as he is — and for having had numerous problems with women in the past. His online sexual harassment does not appear to warrant this kind of punishment, not even by MIT. This was a long time coming and they got rid of him this way. Emails destroy careers. Sorry to say. I feel sorry for Walter too for lacking the insight to treat others better and that he did this to himself.

That was on December 10th, the day after the news broke. I read the comment thread a few days later, absorbed the discussion, and moved on.

So I was surprised the other night by Conor Friedersdorf’s The Blog Comment That Achieved an Internet Miracle, inspired by that very same comment thread. When I’d last checked in, the Aronson thread ended at about comment #75. The comment to which Friedersdorf refers was #171, posted on December 14.

It would be insane to add many more words to the outpouring that followed the now-infamous Comment #171, both on Aronson’s blog and elsewhere. So instead I’ll just add a couple of pictures.

Contributors by number of comments:

Contributors by number of bytes:

What these charts show is that two people dominate the thread which, by the other night, had grown to over 600 comments. There’s Scott Aronson, the author of the blog, who in the two weeks leading up to Christmas wrote 107 comments adding up to about 30,000 words (assuming an average word length of 5 characters). And there’s Amy, who over those same two weeks wrote 82 comments adding up to about 36,000 words.

I can’t begin to summarize the discussion, so I’ll just agree with Conor Friedersdorf’s assessment:

Aaronson and his interlocutors transformed an obscure, not-particularly-edifying debate into a broad, widely read conversation that encompassed more earnest, productive, revelatory perspectives than I’d have thought possible. The conversation has already captivated a corner of the Internet, but deserves wider attention, both as a model of public discourse and a window into the human experience.

There were many interlocutors, but one in particular stood head and shoulders above the crowd: Amy. How often is she mentioned in three widely-cited blog posts about the Comment 171 affair? Let’s look.

0: http://www.theatlantic.com/politics/archive/2015/01/the-blog-comment-that-achieved-an-internet-miracle/384539/ (Conor Friedersdorf)

0: http://www.newstatesman.com/laurie-penny/on-nerd-entitlement-rebel-alliance-empire (Laurie Penny)

0: http://slatestarcodex.com/2015/01/01/untitled/ (Scott Alexander)

Another Internet miracle!

A network of neighbors

A new acquaintance here in Santa Rosa recommended Nextdoor, a service that describes itself as “the private social network for your neighborhood.” Yet another social network? I know. Back in 2007 Gary McGraw nailed the problem of social network fatigue. “People keep asking me to join the LinkedIn network,” he said, “but I’m already part of a network, it’s called the Internet.”

Nevertheless, I joined. We’re new in town, and I don’t want to let my antipathy to walled gardens get in the way of making useful connections. If you haven’t seen Nextdoor it’s because you haven’t joined it. Nextdoor resembles Facebook in many ways. But it’s only visible after you sign up, and you can only do that by proving residence in a neighborhood.

The signup protocol is intriguing:

The postcard method seems safest but I didn’t want to wait. The credit card method is immediate but I dislike using my card that way. So I tried the phone method. You’re asked to provide a phone number that’s billed to your residence, then the phone receives a code you use to complete the signup.

How did the site get my service provider, AT&T, to confirm that my phone’s billing address matches the one I was claiming on Nextdoor? Beats me. On reflection that feels as creepy as identifying to a social network with a credit card, maybe creepier. It’s shame that services like Nextdoor can’t yet verify such claims with identity providers that we choose for the purpose — banks for example, or state governments. But I digress.

Once you sign in, Nextdoor begs you in the usual ways to help grow the network. It prompts you to upload contacts who will be targets for email invitations, and offers a $25 Amazon Gift Card if you’ll post an invitation link on Facebook or Twitter. But there are some uniquely local alternatives too. Nextdoor will send postcards to nearby households, and help you make flyers you can post around the neighborhood.

Nextdoor’s map of my neighborhood reports that 51 of 766 households are claimed by registered users. A progress bar shows that’s 7% of the total neighborhood saturation to which it aspires. The map is a patchwork quilt of claimed addresses, shown in green, and ones yet to be assimilated, shown in pink.

The neighborhood directory lists people who live at the claimed addresses, it links to their profiles, and it offers to send direct messages to them. Local chatter appears on the neighborhood wall and is what you’d expect: a filing cabinet is available for $85, a neighborhood watch meeting will be held next month.

This social network is private in an interesting way. The zone of privacy is defined by the neighborhood boundary. You can most easily find and interact with others within that zone. But you’re also made aware of activities in the wider zone of nearby neighborhoods. Maps of those neighborhoods aren’t as detailed. But you can see posts from people in nearby neighborhoods, communicate with them, and discover them by searching for things they’ve said.

Our neighborhood is near Santa Rosa’s downtown. Nextdoor considers fifteen others, comprising much of the downtown core, to be nearby neighborhoods. You can choose to communicate with nearby neighbors or not. If you do, you reveal less about yourself than to immediate neighbors. It’s a clever design that encourages people to explore the boundaries between what’s public and what’s private, to realize how the online world renders such distinctions fluid and relative, and to learn to behave accordingly.

None of this will appeal to everyone, much less to millenials like Carmen DeAmicis who covers social media for Gigaom. But in a recent Gigaom post she explains why she suddenly found Nextdoor compelling:

Twenty-somethings in urban areas by-and-large don’t have kids, their lives don’t revolve around their home and they know their neighbors hardly, if at all. So even though I covered Nextdoor, I never felt compelled to actually become a user.

That changes today. Nextdoor has introduced a new element to its application that makes it a must-use network, even for the disinterested younger generations. It has started partnering with police and fire departments across the country — in 250 cities initially, with more to come — to use Nextdoor to communicate about emergencies and safety issues with local residents.

Given that Nextdoor sings a familiar tune — “We will NOT require members to pay to use Nextdoor and we will not sell users’ private information to other companies” — that’s a plausible business model. But to partner cities it’s yet another channel of communication to keep track of. And to citizens it’s yet another fragment of online identity.

Cities need to engage with people as individuals, members of interest groups, and residents of neighborhoods, in multi-faceted ways that reflect personal preferences, local customs, and generational trends. Nextdoor is interesting and useful, but I would rather see neighborhood social networks arise as organically online as they do on the ground. There isn’t an app for that, but there is a network, or rather there will be. In that network you’ll choose various parties to certify claims about aspects of your identity to various other parties. Those claims will define your affiliations to various geographic and interest groups. Your communication within those groups will flow through channels that you specify. What is that network? We’ll call it the Internet, and we’ll all be neighbors there.

A federated Wikipedia

Writing for the Chronicle of Higher Education in 2012, Timothy Messer-Kruse described his failed efforts to penetrate Wikipedia’s gravitational field. He begins:

For the past 10 years I’ve immersed myself in the details of one of the most famous events in American labor history, the Haymarket riot and trial of 1886. Along the way I’ve written two books and a couple of articles about the episode. In some circles that affords me a presumption of expertise on the subject. Not, however, on Wikipedia.

His tale of woe will be familiar to countless domain experts who thought Wikipedia was the encyclopedia anyone can edit but found otherwise. His research had led to the conclusion that a presumed fact, often repeated in the scholarly literature, was wrong. Saying so triggered a rejection based on Wikipedia’s policy on reliable sources and undue weight. Here was the ensuing exchange:

Explain to me, then, how a ‘minority’ source with facts on its side would ever appear against a wrong ‘majority’ one?” I asked the Wiki-gatekeeper. He responded, “You’re more than welcome to discuss reliable sources here, that’s what the talk page is for. However, you might want to have a quick look at Wikipedia’s civility policy.

(You can relive his adventure by visiting this revision of the article’s talk page and clicking the Next edit link a half-dozen times. You have to dig to find backstories like this one. But to Wikipedia’s credit, they are preserved and can be found.)

Timothy Messer-Kruse’s Wikipedia contributions page summarizes his brief career as a Wikipedia editor. He battled the gatekeepers for a short while, then sensibly retreated. As have others. In The Closed, Unfriendly World of Wikipedia, Internet search expert Danny Sullivan blogged his failed effort to offer some of his expertise. MIT Technology Review contributor Tom Simonite, in The Decine of Wikipedia, calls Wikipedia “a crushing bureaucracy with an often abrasive atmosphere that deters newcomers” and concludes:

Today’s Wikipedia, even with its middling quality and poor representation of the world’s diversity, could be the best encyclopedia we will get.

That would be a sad outcome. It may be avoidable, but only if we take seriously the last of Wikipedia’s Five pillars. “Wikipedia has no firm rules,” that foundational page says, it has “policies and guidelines, but they are not carved in stone.” Here is the policy that most desperately needs to change: Content forking:

A point of view (POV) fork is a content fork deliberately created to avoid neutral point of view guidelines, often to avoid or highlight negative or positive viewpoints or facts. All POV forks are undesirable on Wikipedia, as they avoid consensus building and therefore violate one of our most important policies.

That policy places Wikipedia on the wrong side of history. Not too long ago, we debated whether a distributed version control system (DVCS) could possibly work, and regarded forking an open source project as a catastrophe. Now GitHub is the center of an open source universe in which DVCS-supported forking is one of the gears of progress.

Meanwhile, as we near the 20th anniversary of wiki software, its inventor Ward Cunningham is busily reimagining his creation. I’ve written a lot lately about his new federated wiki, an implementation of the wiki idea that values a chorus of voices. In the federated wiki you fork pages of interest and may edit them. If you do, your changes may or may not be noticed. If they are noticed they may or may not be merged. But they belong to the network graph that grows around the page. They are discoverable.

In Federated Education: New Directions in Digital Collaboration, Mike Caulfield offers this key insight about federated wiki:

Wiki is a relentless consensus engine. That’s useful.

But here’s the thing. You want the consensus engine, eventually. But you don’t want it at first.

How can we ease the relentlessness of Wikipedia’s consensus engine? Here’s a telling comment posted to Timothy Messer-Kruse’s User talk page after his Chronicle essay appeared:

Great article. Next time just go ahead and make all of your changes in one edit, without hesitation. If you are reverted, then make a reasonable educated complaint in the talk page of the article (or simply write another article for the Chronicle, or a blog post). Other people with more, eh, “wikiexperience” will be able to look at your edit, review the changes, and make them stand.

To “write another article for the Chronicle, or a blog post” is, of course, a way of forking the Wikipedia article. So why not encourage that? There aren’t an infinite number of people in the world who have deep knowledge of the Haymarket affair and are inclined to share it. The network graph showing who forked that Wikipedia article, and made substantive contributions, needn’t be overwhelming. Timothy Messer-Kruse’s fork might or might not emerge as authoritative in the judgement of Wikipedia but also of the world. If it did, Wikipedia might or might not choose to merge it. But if the consensus engine is willing to listen for a while to a chorus of voices, it may be able to recruit and retain more of the voices it needs.

Thoughts in motion

In Federated Wiki for teaching and learning basic composition I speculated about a FedWiki plugin that would surface the version history of individual paragraphs within an essay. Over the weekend I prototyped that plugin and you can see it in action here. The paragraph that begins with “The relevance to SFW” is the one that I featured in the original blog post. On the wiki, in a forked copy of the Kate Bowles essay that prompted my inquiry, I’ve injected a plugin that lists paragraphs that have changed at least once since originally written, and that displays the version-to-version changes for each. On that page the plugin shows up in light yellow. If you click the + that precedes “The relevance to SFW” there, you should see the same record of changes I reported in the blog post. It looks like this:

This view improves on the mock-up shown in the original blog post, adding color-coded highlighting of version-to-version differences. I think it’s a striking illustration of how a thought evolves through a sequence of written snapshots. It reminds me of Michael Rubinstein’s astonishing TED talk on a video processing technique that reveals and amplifies normally undetectable color change and motion.

If Kate had been using a conventional writing tool, it would be much harder to observe what we see here. But in FedWiki a paragraph is special. It has its own version history. Every time you open up a paragraph to change it, that change is recorded. To the extent there’s a correspondence between paragraphs and thoughts — and in prose that is often the case — FedWiki intrinsically enables the study of those thoughts in motion.

When computers make visible what was formerly invisible there is always an initial rush of excitement. Then the question becomes: Is this useful? And if so, in what ways?

I can envision two primary uses of this technique. First, for writers. We all compose differently, and not everyone will want to be able to replay the changes to an individual paragraph. But if you do want to, conventional tools aren’t much help. In Google Docs, for example, you can roll through versions of a whole document but it’s not easy to focus on how a single paragraph changes.

A second use, as suggested in the original post, is for teachers of writing and their students. In other domains, such as music, computers have become powerful tools for studying finished compositions. Adrian Holovaty’s Soundslice, for example, makes YouTube performances accessible to study and annotation. In that case there’s nothing hidden, the tool’s job is to slow things down and provide a synchronized framework for annotation. But what if you wanted to study the process of musical composition? Then you’d want the composition to occur in an environment that records changes in chunks of varying granularity that correspond to the structure of the piece.

Because FedWiki naturally divides writing into paragraph chunks, it enables us to see how paragraph-sized thoughts evolve. But a FedWiki page is not only a sequence of paragraphs. Other kinds of objects can be injected into the page by plugin that manage, for example, data sets and small scripts written in domain-specific languages. These things have their own version histories too.

Most knowledge workers, with the notable exception of software developers, don’t yet use tools that take version control for granted. That will change. The artifacts produced by knowledge work are simply too valuable to be managed any other way. As versioning tools evolve for other disciplines, we have the opportunity to rethink what those tools can do. I hope we’ll do that with sensitivity to the natural granularity of the material. In many cases, whole documents aren’t the right chunks.

Even in software development, of course, we are still working with document-sized chunks. Compilers know that programs are made of modules and functions, but editing tools don’t track changes that way and as a result GitHub can’t directly show us the history of an individual function. That would be useful for the same reasons I’ve mentioned. It would help programmers reflect on their own work, and enable teachers to show students more about how code is written and how it evolves.

Tools for structured writing aren’t a new idea, of course. There are reasons why they haven’t caught on. But FedWiki reminds us that there are also reasons to hope they will.

How Federated Wiki neighborhoods grow and change

Federated Wiki sites form neighborhoods that change dynamically as you navigate FedWiki space. Sites that are within your current neighborhood are special in two ways: you can link to them by names alone (versus full URLs), and you can search them.

Here’s one neighborhood I can join.

A row of flags (icons) in the bottom right corner of the screen (1) indicates that there are five sites in this neighborhood: my own and four others. The number next to the search box in the bottom middle (2) says that 772 pages can be searched. That number is the sum of all the pages in the neighborhood.

From each site in the neighborhood, FedWiki retrieves a summary called the sitemap. It is a list of all the pages on the site. Each item in the list has the page’s title, date, and complete first paragraph (which might be very short or very long). FedWiki’s built-in search uses sitemaps which means that it only sees the titles and first paragraphs of the pages in your neighborhood.

Here are the sites in this neighborhood:

  1. forage.ward.fed.wiki.org
  2. jon.sf.fedwikihappening.net
  3. sites.fed.wiki.org
  4. video.fed.wiki.org
  5. ward.fed.wiki.org

You can find these names by hovering over the row of flags. If you are technical you might also want to observe them in a JavaScript debugger. In this picture, I used Control-J in Chrome to launch the debugger, then clicked into the Console tab, then typed the name of the JavaScript variable that represents the neighborhood: wiki.neighborhood.

Why are these five sites in my neighborhood? It’s obvious that my own site, jon.sf.fedwikihappening.net, belongs. And since I’ve navigated to a page on forage.ward.fed.wiki.org, it’s not suprising to find that site in my neighborhood too. But what about the other three? Why are they included?

The answer is that Ward’s page includes references to sites.fed.wiki.org, video.fed.wiki.org, and ward.fed.wiki.org. A FedWiki reference looks like a paragraph, but its blue tint signals that it’s special. Unlike a normal paragraph, which you inject into the page using the HTML or Markdown plugin, a reference is injected using the Reference plugin. It’s a dynamic element that displays the flag, the page name, and synopsis (first paragraph) of the referenced page. It also adds that page’s origin site to the neighborhood.

Two of the five sites in this example neighborhood — jon.sf.fedwikihappening.net and forage.ward.fed.wiki.org — got there directly by way of navigation. The other three got there indirectly by way of references.

To add a reference to one of your own pages, you click the + symbol to add a factory, drag the flag (or favicon) of a remote FedWiki page, and drop it onto the factory.

To illustrate, I’ll start with a scratch page that has a factory ready to accept a drop.

In a second browser tab, I’ll navigate to forage.ward.fed.wiki.org’s Ward Cunningham page, the one with the three references we saw above. Then I’ll drag that page’s favicon into the first browser tab and drop it onto the factory. Dragging between browser tabs may be unfamiliar to you. It was to me as well, actually. But it’s a thing.

The setup in this example is:

Tab 1: http://jon.sf.fedwikihappening.net/view/welcome-visitors/view/scratch

Tab 2: http://forage.ward.fed.wiki.org/view/ward-cunningham

Here is the result:

How many sites are in this neighborhood? When I did this experiment, I predicted either 2 or 5. It would be 2 if the neighborhood included only my site and the origin of the referenced page. It would be 5 if FedWiki included, in addition, sites referenced on the referenced page. Things aren’t transitive in that way, it turns out, so the answer is 2.

Except that it isn’t. It’s 3! Look at the row of flags in the bottom right corner. There are three of them: jon.sf.fedwikihappening.net, forage.ward.fed.wiki.org, and mysteriously, fedwikihappening.rodwell.me. That’s Paul Rodwell’s site. How did he get into this neighborhood?

This closeup of the journal will help explain the mystery. The page was forked 5 days ago.

We can view the source of the page to find out more.

And here’s the answer. Early in the life of my scratch page I forked Paul Rodwell’s scratch page from fedwikihappening.rodwell.me.

So we’ve now discovered a third way to grow your neighborhood. First by navigating to remote pages directly. Second by including references to remote pages. And third by forking remote pages.

FedWiki for collaborative analysis of data

A FedWiki page presents one or more wiki pages side by side. This arrangement is called the lineup. During interactive use of FedWiki the lineup grows rightward as you navigate the federation. But you can also compose a lineup by forming an URL that describes a purposeful arrangement of wiki pages. In Federated Wiki for teaching and learning basic composition I composed two lineups. The first compares two versions of a page on Kate Bowles’ FedWiki site. The second compares two versions of that page from two different sites: mine and Kate’s. With these two lineups I’m exploring the notion that FedWiki could be a writers’ studio in which students watch their own paragraphs evolve, and also overlay suggestions from teachers (or other students).

In that example the order of wiki pages in the lineup isn’t important. You can compare versions left-to-right or right-to-left. But here’s another example where left-to-right sequence matters:

Link: Favorite Broccoli Recipes

URL: http://jon.sf.fedwikihappening.net/view/italian-broccoli/view/broccoli-fried-with-sesame-and-raspberry/view/favorite-broccoli-recipes

Rendering:

The tables shown in these wiki pages are made by a data plugin that accumulates facts and performs calculations. FedWiki has explored a number of these data plugins. This one implements the little language that you can see in these views of the text that lives in those embedded plugins:

On the Italian Broccoli page:

5 (calories) per (garlic clove)
200 (calories) per (bunch of broccoli)
SUM Italian Broccoli (calories)

On the Broccoli Fried With Sesame and Raspberry page:

100 (calories) per (tbsp sesame seed oil)
34 (calories) per (100 grams broccoli)

And:

3 (tbsp sesame seed oil)
SUM (calories)
1 (100 grams broccoli)
SUM Broccoli Fried With Sesame Oil (calories)

On the Favorite Broccoli Recipes page:

Italian Broccoli (calories)

And:

Broccoli Fried With Sesame Oil (calories)

Other plugins implement variations on this little language, and it’s surprisingly easy to create new ones. What I’m especially drawing attention to here, though, is that the lineup of wiki pages forms a left-to-right pipeline. Facts and calculations flow not only downward within a wiki page, but also rightward through a pipeline of wiki pages.

And that pipeline, as we’ve seen, can be composed of pages from one site, or of pages drawn from several sites. I could provide one set of facts, you could provide an alternative set of facts, anyone could build a pipeline that evaluates both. It’s a beautiful way to enable the collaborative production and analysis of data.

Federated Wiki for teaching and learning basic composition

The FedWikiHappening has mainly explored Federated Wiki as an environment for collaborative writing. But the underlying software is rich with unexplored capability. It is, among many other possible uses, a great platform for the teaching and learning of basic writing skills.

Every page in FedWiki is backed by two data structures. The story is a sequence of paragraphs. The journal is a sequence of actions that add, edit, move, or delete paragraphs. Because editing is paragraph-oriented, the progressive rewriting of a paragraph is recorded in the journal.

I once taught an introductory writing class to undergraduates. Part of my method was to awaken students to the notion that paragraphs can and should evolve, and that it’s useful to observe and discuss that evolution. In FedWiki the evolution of a paragraph is not directly visible, but it’s available just below the surface. Here’s a beautiful example from a Kate Bowles essay called Sentences that get things done. The essay emerged in response to a collaborative riff that ends with Kate’s title. But here let’s watch one paragraph in Kate’s essay grow and change.

1. The relevance to wiki is that vernacular language is both capable of sustaining narrative and does not depend on citation.


2. The relevance to SFW is that vernacular language is both capable of sustaining narrative and does not depend on citation.


3. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation.


4. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed.


5. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed, and so becomes a practice of creating sentences that get things done, rather than further intensification of the spectacle of heroic individuals doing things.


6. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed, and so becomes a practice of collaboratively creating sentences that get things done, rather than further intensification of the spectacle of heroic individuals doing things.


7. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed, and so becomes a practice of collaboratively creating sentences that get things done, as a counterpoint to the intensification of heroic individuals doing things.


8. The relevance to SFW is that vernacular language is both capable of sustaining and amplifying personal narrative and yet does not depend on authorial celebrity or citation. Vernacular language is available to be borrowed, forked, repurposed, and so becomes a practice of collaboratively creating sentences that get things done.

Version 4 might have been a keeper. But something propelled Kate to work through versions 5, 6, and 7. In the final version we see what she was reaching for: a way to land on the sentence that is both the essay’s title and a reference to the context from which the essay arose.

Any kind of web client software, running in the browser or in the cloud, could access that essay’s journal and surface that paragraph history. The FedWiki API (application programming interface) is simple and universal: just subtract /view from a FedWiki URL and append .json to the name of a page. So, for example:

Kate’s page: http://kate.au.fedwikihappening.net/view/sentences-that-get-things-done

The API for Kate’s page: http://kate.au.fedwikihappening.net/sentences-that-get-things-done.json

We can also construct URLs that arrange versions side by side. Here’s a FedWiki lineup that arranges two versions of the paragraph side by side and in context:

http://kate.au.fedwikihappening.net/view/sentences-that-get-things-done_rev26/view/sentences-that-get-things-done_rev36

Now imagine that I’m the teacher, Kate is the student, I’ve forked Kate’s essay, and I’ve written version 8 as an example for Kate. Here’s an URL that arranges her version alongside mine:

http://kate.au.fedwikihappening.net/view/sentences-that-get-things-done_rev26/jon.sf.fedwikihappening.net/sentences-that-get-things-done_rev36

I would love to help build tools that mine FedWiki’s latent ability to support the teaching and learning of prose composition. And I would equally love using those tools to facilitate that teaching and learning.

Individual voices in the Federated Wiki chorus

In recent days I’ve been immersed in the Federated Wiki Happening, a group exploration of Ward Cunningham’s Smallest Federated Wiki (SFW). When I first saw what Ward was up to, nearly a year ago, I resisted the temptation to dive in because I knew it would be a long and deep dive that I couldn’t make time for. But when Mike Caulfield brought together a diverse group of like-minded scholars for the #fedwikihappening, I had the time and took the plunge. It’s been a joyful experience that reminds me of two bygone eras. The first was the dawn of the web, when I built the BYTE website and explored the Internet’s precursors to today’s social software. The second was the dawn of the blogosphere, when I immersed myself in Radio UserLand and RSS.

During both of those eras I participated in online communities enaged in, among other things, the discovery of emergent uses of the networked software that enabled those communities to exist. The interplay of social and technological dynamics was exciting in ways I’d almost forgotten. This week, the FedWikiHappening took me there again.

I want to explain why but, as Mike says today, so much has happened so quickly that it’s hard to know where to begin. For now, I’ll choose a single narrative thread: identity.

SFW inverts the traditional wiki model, which enables many authors to work on a canonical page. In SFW there is no canonical page. We all create our own pages and edit them exclusively. But we can also copy pages from others, and make changes. Others may (or may not) notice those changes, and may (or may not) merge the changes.

In this respect SFW resembles GitHub, and its terminology — you “fork” a page from an “origin site” — invites the comparison. But SFW is looser than GitHub. What GitHub calls a pull request, for example, isn’t (yet) a well-developed feature of SFW. And while attribution is crystal-clear in GitHub — you always know who made a contribution — it is (by design) somewhat vague in SFW. In the Chorus of Voices that Ward envisions, individual voices are not easy to discern.

That notion was hard for some of us in The Happening, myself included, to swallow. In SFW we were represented not as avatars with pictures but as neutral “flags” made of color gradients. Identity was Discoverable But Not Obvious.

Then Alex North cracked the code. He read through the FedWiki sources, found the hook for uploading the favicon that serves as SFW’s flag/avatar, and worked out a procedure for using that hook to upload an image.

The next day I worked out a Windows-friendly variant of Alex’s method and uploaded my own image. Meanwhile a few other Happening participants used Alex’s method to replace their colored gradients with photos.

The next day Mike Caulfield bowed to the will of the people and uploaded a batch of photos on behalf of participants unable to cope with Alex’s admittedly geeky hack. Suddenly the Happening looked more like a normal social network, where everyone’s contributions have identifying photos.

That was a victory, but not an unqualified one.

It was a victory in part because Alex showed the group that SFW is web software, and like all web software is radically open to unintended uses. Also, of course, because we were able to alter the system in response to a perceived need.

And yet, we may have decided too quickly not to explore a mode of collaboration that favors the chorus over the individual voice. Can we work together effectively that way, in a federated system that ultimately gives us full control of our own data? That remains an open question for me, one of many that the Happening has prompted me to ask and explore.

TypeScript Successes and Failures

My last post buried the lead, so I’ll hoist it to the top here: I’ve left Microsoft. While I figure out what my next gig will be, I’ll be doing some freelance writing and consulting. My first writing assignment will be an InfoWorld feature on TypeScript. It’s an important technology that isn’t yet well understood or widely adopted. I made two efforts to adopt it myself. The first, almost a year ago, didn’t stick. The second, a few weeks ago, did.

I’ll reflect on those experiences in the article. But I’m also keen to mine other perspectives on why TypeScript adoption fails or succeeds. And I’m particularly interested to hear about experiences with TypeScript toolchains other than Visual Studio. If you have perspectives and experiences to share, please drop a note here or to jon at jonudell.info.

Skype Translator will (also) be a tool for language learners

When I saw this video of Skype Translator I realized that beyond just(!) translation, it will be a powerful tool for language learning. Last night I got a glimpse of that near future. Our next door neighbor, Yolanda, came here from Mexico 30 years ago and is fluently bilingual. She was sitting outside with her friend, Carmen, who speaks almost no English. I joined them and tried to listen to their Spanish conversation. I learned a bit of Spanish in high school but I’ve never been conversational. Here in Santa Rosa I’m surrounded by speakers of Spanish, it’s an opportunity to learn, and Yolanda — who has worked as a translator in the court system — is willing to help.

I find myself on parallel tracks with respect to my learning of two different languages: music and Spanish. In both cases I’ve historically learned more from books than by ear. Now I want to put myself into situations that force me to set the books aside, listen intently, and then try to speak appropriately. I can use all the help I can get. Luckily we live in an era of unprecedented tool support. On the musical front, I’ve made good use of Adrian Holovaty’s SoundSlice, a remarkable tool for studying and transcribing musical performances it pulls from YouTube. I haven’t used SoundSlice much for annotation, because I’m trying to develop my ear and my ability to converse musically in realtime. But its ability to slow down part of a tune, and then loop it, has been really helpful in my efforts to interact with real performances.

I suspect that’s why Skype Translator will turn out to be great for language learning. Actually I’m sure that will happen, and here’s why. Last night I showed the Skype Translator video to Yolanda and Carmen. Neither is remotely tech-savvy but both instantly understood what was happening. Yolanda marveled to see machine translation coming alive. Carmen, meanwhile, was transfixed by the bilingual exchange. And when she heard the English translation of a Spanish phrase, I could see her mouthing the English words. I found myself doing the same for the Spanish translation of English phrases.

That’s already a powerful thing, and yet we were only observers of a conversation. When we can be participants, motivated to communicate, the service won’t just be a way to speak across a language gap. It’ll be a way to learn one another’s languages.

No disclosure is needed here, by the way, because I’m a free agent for now. My final day with Microsoft was last Friday. In the end I wasn’t able to contribute in the ways I’d hoped I could. But great things are happening there, and Skype Translator is only one of the reasons I’m bullish on the company’s future.

Human/machine partnership for problems otherwise Too Hard

My recent post about redirecting a page of broken links weaves together two different ideas. First, that the titles of the articles on that page of broken links can be used as search terms in alternate links that lead people to those articles’ new locations. Second, that non-programmers can create macros to transform the original links into alternate search-driven links.

There was lots of useful feedback on the first idea. As Herbert Van de Sompel and Michael Nelson pointed out, it was a really bad idea to discard the original URLs, which retain value as lookup keys into one or more web archives. Alan Levine showed how to do that with the Wayback Machine. That method, however, leads the user to sets of snapshots that don’t consistently mirror the original article, because (I think) Wayback’s captures happened both before and after the breakage.

So for now I’ve restored the original page of broken links, alongside the new page of transformed links. I’m grateful for the ensuing discussion about ways to annotate those transformed links so they’re aware of the originals, and so they can tap into evolving services — like Memento — that will make good use of the originals.

The second idea, about tools and automation, drew interesting commentary as well. dyerjohn pointed to NimbleText, though we agreed it’s more suited to tabular than to textual data. Owen Stephens reminded me that the tool I first knew as Freebase Gridworks, then Google Refine, is going strong as OpenRefine. And while it too is more tabular than textual, “the data is line based,” he says, “and sort of tabular if you squint at it hard.” In Using OpenRefine to manipulate HTML he presents a fully-worked example of how to use OpenRefine to do the transformation I made by recording and playing back a macro in my text editor.

Meanwhile, on Twitter, Paul Walk and Les Carr and I were rehashing the old permathread about coding for non-coders.

The point about MS Word styles is spot on. That mechanism asks people to think abstractly, in terms of classes and instances. It’s never caught on. So with my text-transformation puzzle, Les suggests. Even with tools that enable non-coders to solve the puzzle, getting people across the cognitive threshold is Too Hard.

While mulling this over, I happened to watch Jeremy Howard’s TED talk on machine learning. He demonstrates a wonderful partnership between human and machine. The task is to categorize a large set of images. The computer suggests groupings, the human corrects and refines those groupings, the process iterates.

We’ve yet to inject that technology into our everyday productivity tools, but we will. And then, maybe, we will finally start to bridge the gap between coders and non-coders. The computer will watch the styles I create as I write, infer classes, offer to instantiate them for me, and we will iterate that process. Similarly, when I’m doing a repetitive transformation, it will notice what’s happening, infer the algorithm, offer to implement it for me, we’ll run it experimentally on a sample, then iterate.

Maybe in the end what people will most need to learn is not how to design stylistic classes and instances, or how to write code that automates repetitive tasks, but rather how to partner effectively with machines that work with us to make those things happen. Things that are Too Hard for most living humans and all current machines to do on their own.

Where’s the IFTTT for repetitive manual text transformation?

While updating my home page today, I noticed that that the page listing my InfoWorld articles had become a graveyard of broken links. The stuff is all still there, but at some point the site switched to another content management system without redirecting old URLs. This happens to me from time to time. It’s always annoying. In some cases I’ve moved archives to my own personal web space. But I prefer to keep them alive in their original contexts, if possible. This time around, I came up with a quick and easy way to do that. I’ll describe it here because it illustrates a few simple and effective strategies.

My listing page looks like this:

<p><a href=”http://www.infoworld.com/article/06/11/15/47OPstrategic_1.html”>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

<p><a href=”http://www.infoworld.com/article/06/11/08/46OPstrategic_1.html”>Web apps, just give me the data | Column | 2006-11-08</a></p>

It’s easy to see the underlying pattern:

LINK | CATEGORY | DATE

When I left InfoWorld I searched the site for everything I’d written there and made a list, in the HTML format shown above, that conformed to the pattern. Today I needed to alter all the URLs in that list. My initial plan was to search for each title using this pattern:

site:infoworld.com “jon udell” “TITLE”

For example, try this in Google or Bing:

site:infoworld.com “jon udell” “xquery and the power of learning by example”

Either way, you bypass the now-broken original URL (http://www.infoworld.com/article/06/11/15/47OPstrategic_1.html) and are led to the current one (http://www.infoworld.com/article/2660595/application-development/xquery-and-the-power-of-learning-by-example.html)

The plan was then to write a script that would robotically perform those searches and extract the current URL from each result. But life’s short, I’m lazy, and I realized a couple of things. First, the desired result is usually but not always first, so the script would need to deal with that. Second, what if the URLs change yet again?

That led to an interesting conclusion: the search URLs themselves are good enough for my purposes. I just needed to transform the page of links to broken URLs into a page of links to title searches constrained to infoworld.com and my name. So that’s what I did, it works nicely, and the page is future-proofed against future URL breakage.

I could have written code to do that transformation, but I’d rather not. Also, contrary to a popular belief, I don’t think everyone can or should learn to write code. There are other ways to accomplish a task like this, ways that are easier for me and — more importantly — accessible to non-programmers. I alluded to one of them in A web of agreements and disagreements, which shows how to translate from one wiki format to another just by recording and using a macro in a text editing program. I used that same strategy in this case.

Of course recording a macro is a kind of coding. It’s tricky to get it to do what you intend. So here’s a related strategy: divide a complex transformation into a series of simpler steps. Here are the steps I used to fix the listing page.

Step 1: Remove the old URLs

The old URLS are useless clutter at this point, so just get rid of them.

old: <p><a href=”http://www.infoworld.com/article/06/11/15/47OPstrategic_1.html”>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

new: <p><a href=””>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

how: Search for href=”, mark the spot, search for “>, delete the highlighted selection between the two search targets, go to the next line.

Step 2: Add query templates

We’ve already seen the pattern we need: site:infoworld.com “jon udell” “TITLE”. Now we’ll replace the empty URLs with URLs that include the pattern. To create the template, search Google or Bing for the pattern. (I used Bing but you can use Google the same way.) You’ll see some funny things in the URLs they produce, things like %3A and %22. These are alternate ways of representing the equals sign and the double quote. They make things harder to read, but you need them to preserve the integrity of the URL. Copy this URL from the browser’s location window to the clipboard.

old: <p><a href=””>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

new: <p><a href=”http://www.bing.com/search?q=site%3Ainfoworld.com+%22jon+udell%22+%22%5BTITLE%5D%22″>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

how: Copy the template URL to the clipboard. Then for each line, search for href=””, put the cursor after the first double quote, paste, and go to the next line.

Step 3: Replace [TITLE] in each template with the actual title

old: <p><a href=”http://www.bing.com/search?q=site%3Ainfoworld.com+%22jon+udell%22+%22%5BTITLE%5D%22″>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

new: <p><a href=”http://www.bing.com/search?q=site%3Ainfoworld.com+%22jon+udell%22+%22XQuery and the power of learning by example%22″>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

how: For each line, search for >”, mark the spot, search for |, paste, copy the highlighted section between the two search targets, search for [TITLE], put the cursor at [, delete the next 7 characters, paste from the clipboard.

Now that I’ve written all this down, I’ll admit it looks daunting, and doesn’t really qualify as a “no coding required” solution. It is a kind of coding, to be sure. But this kind of coding doesn’t involve a programming language. Instead you work out how to do things interactively, and then capture and replay those interactions.

I’ll also admit that, even though word processors like Microsoft Word and LibreOffice can do capture and replay, you’ll be hard pressed to pull off a transformation like this using those tools. They’re not set up to do incremental search, or switch between searching and editing while recording. So I didn’t use a word processor, I used a programmer’s text editor. Mine’s an ancient one from Lugaru Software, there are many others, all of which will be familiar only to programmers. Which, of course, defeats my argument for accessibility. If you are not a programmer, you are not going to want to acquire and use a tool made for programmers.

So I’m left with a question. Are there tools — preferably online tools — that make this kind of text transformation widely available? If not, there’s an opportunity to create one. What IFTTT is doing for manual integration of web services is something that could also be done for manual transformation of text. If you watch over an office worker’s shoulder for any length of time, you’ll see that kind of manual transformation happening. It’s a colossal waste of time (and source of error). I could have spent hours reformatting that listing page. Instead it took me a few minutes. In the time I saved I documented how to do it. I wasn’t able to give you a reusable and modifiable online recipe, but that’s doable and would be a wonderful thing to enable.

Why shouting won’t help you talk to a person with hearing loss

I’ve written a few posts [1, 2] about my mom’s use of a reading machine to compensate for macular degeneration, and I made a video that shows the optimal strategy for using the machine. We’re past the point where she can get any benefit from the gadget, though. She needs such extreme magnification that it’s just not worth it any more.

So she’s more dependent than ever on her hearing. Sadly her hearing loss is nearly as profound as her vision loss, and hearing aids can’t compensate as well as we wish. She’s still getting good mileage out of audiobooks, and also podcasts which she listens to on MP3 players that I load up and send her. The clear and well-modulated voice of a single speaker, delivered through headphones that block out other sound, works well for her. But in real-world situations there are often several voices, not clear or well-modulated, coming from different parts of the room and competing with other ambient noise. She depends on hearing aids but as good as they’ve gotten, they can’t yet isolate and clarify those kinds of voices.

One of the best ways to communicate with my mom is to speak to her on the phone. That puts the voice directly in her ear while the phone blocks other sounds. And here’s a pro tip I got from the audiologist I visited today. If she removes the opposite hearing aid, she’ll cut down on ambient noise in the non-conversational ear.

In person, the same principle applies. Put the voice right into her ear. If I lean in and speak directly into her ear, I can speak in a normal voice and she can understand me pretty well. It’s been hard to get others to understand and apply that principle, though. People tend to shout from across the room or even from a few feet away. Those sounds don’t come through as clearly as sounds delivered much more softly directly into the ear. And shouting just amps up the stress in the room, which nobody needs.

Lately, though, the voice-in-the-ear strategy — whether on the phone or in person — had been failing us. We had thought maybe the hearing aids needed be cleaned, but that wasn’t the problem. She’s been accidentally turning down the volume! There’s a button on each hearing aid that you tap to cycle through the volume settings. I don’t think mom understood that, and I know she can’t sense if she touches the button while reseating the device with her finger. To compound the problem, the button’s action defaults to volume reduction. If it went the other way she might have been more likely to notice an accidental change. But really, given that she’s also losing dexterity, the volume control is just a useless affordance for her.

Today’s visit to the audiologist nailed the problem. When he hooked the hearing aids up to his computer and read their logs(!), we could see they’d often been running at reduced volume. On her last visit he’d set them to boot up at a level we’ll call 3 on a scale of 1 to 5. That’s the level he’d determined was best for her. He’d already had an inkling of what could go wrong, because on that visit he’d disabled the button on the left hearing aid. Now both are disabled, and the setting will stick to 3 unless we need to raise it permanently.

Solving that problem will help matters, but hearing aids can only do so much. The audiologist’s digital toolkit includes a simulator that enabled us to hear a pre-recorded sample voice the way my mom hears it. That was rather shocking. The unaltered voice was loud and clear. Then he activated mom’s profile, and the voice faded so low I thought it was gone completely. I had to put my ear right next to the computer’s speaker to hear it at all, and then it was only a low murmur. When there aren’t many hair cells doing their job in the inner ear, it takes a lot of energy to activate the few that still work, and it’s hard apply that energy with finesse.

I’m sure we’ll find ways to compensate more effectively. That won’t happen soon enough for my mom, though. I wonder if the audiologist’s simulator might play a useful role in the meantime. When we speak to a person with major hearing loss we don’t get any feedback about how we’re being heard. It’s easy to imagine a device that would record a range of speech samples, from shouting at a microphone from across the room to shouting at it from a few feet away to speaking softly directly into it. Then the gadget would play those sounds back two ways: first unaltered, then filtered through the listener’s hearing-loss profile. Maybe that would help people realize that shouting doesn’t help, but proper positioning does.