Human/machine partnership for problems otherwise Too Hard

My recent post about redirecting a page of broken links weaves together two different ideas. First, that the titles of the articles on that page of broken links can be used as search terms in alternate links that lead people to those articles’ new locations. Second, that non-programmers can create macros to transform the original links into alternate search-driven links.

There was lots of useful feedback on the first idea. As Herbert Van de Sompel and Michael Nelson pointed out, it was a really bad idea to discard the original URLs, which retain value as lookup keys into one or more web archives. Alan Levine showed how to do that with the Wayback Machine. That method, however, leads the user to sets of snapshots that don’t consistently mirror the original article, because (I think) Wayback’s captures happened both before and after the breakage.

So for now I’ve restored the original page of broken links, alongside the new page of transformed links. I’m grateful for the ensuing discussion about ways to annotate those transformed links so they’re aware of the originals, and so they can tap into evolving services — like Memento — that will make good use of the originals.

The second idea, about tools and automation, drew interesting commentary as well. dyerjohn pointed to NimbleText, though we agreed it’s more suited to tabular than to textual data. Owen Stephens reminded me that the tool I first knew as Freebase Gridworks, then Google Refine, is going strong as OpenRefine. And while it too is more tabular than textual, “the data is line based,” he says, “and sort of tabular if you squint at it hard.” In Using OpenRefine to manipulate HTML he presents a fully-worked example of how to use OpenRefine to do the transformation I made by recording and playing back a macro in my text editor.

Meanwhile, on Twitter, Paul Walk and Les Carr and I were rehashing the old permathread about coding for non-coders.

The point about MS Word styles is spot on. That mechanism asks people to think abstractly, in terms of classes and instances. It’s never caught on. So with my text-transformation puzzle, Les suggests. Even with tools that enable non-coders to solve the puzzle, getting people across the cognitive threshold is Too Hard.

While mulling this over, I happened to watch Jeremy Howard’s TED talk on machine learning. He demonstrates a wonderful partnership between human and machine. The task is to categorize a large set of images. The computer suggests groupings, the human corrects and refines those groupings, the process iterates.

We’ve yet to inject that technology into our everyday productivity tools, but we will. And then, maybe, we will finally start to bridge the gap between coders and non-coders. The computer will watch the styles I create as I write, infer classes, offer to instantiate them for me, and we will iterate that process. Similarly, when I’m doing a repetitive transformation, it will notice what’s happening, infer the algorithm, offer to implement it for me, we’ll run it experimentally on a sample, then iterate.

Maybe in the end what people will most need to learn is not how to design stylistic classes and instances, or how to write code that automates repetitive tasks, but rather how to partner effectively with machines that work with us to make those things happen. Things that are Too Hard for most living humans and all current machines to do on their own.

Where’s the IFTTT for repetitive manual text transformation?

While updating my home page today, I noticed that that the page listing my InfoWorld articles had become a graveyard of broken links. The stuff is all still there, but at some point the site switched to another content management system without redirecting old URLs. This happens to me from time to time. It’s always annoying. In some cases I’ve moved archives to my own personal web space. But I prefer to keep them alive in their original contexts, if possible. This time around, I came up with a quick and easy way to do that. I’ll describe it here because it illustrates a few simple and effective strategies.

My listing page looks like this:

<p><a href=”http://www.infoworld.com/article/06/11/15/47OPstrategic_1.html”>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

<p><a href=”http://www.infoworld.com/article/06/11/08/46OPstrategic_1.html”>Web apps, just give me the data | Column | 2006-11-08</a></p>

It’s easy to see the underlying pattern:

LINK | CATEGORY | DATE

When I left InfoWorld I searched the site for everything I’d written there and made a list, in the HTML format shown above, that conformed to the pattern. Today I needed to alter all the URLs in that list. My initial plan was to search for each title using this pattern:

site:infoworld.com “jon udell” “TITLE”

For example, try this in Google or Bing:

site:infoworld.com “jon udell” “xquery and the power of learning by example”

Either way, you bypass the now-broken original URL (http://www.infoworld.com/article/06/11/15/47OPstrategic_1.html) and are led to the current one (http://www.infoworld.com/article/2660595/application-development/xquery-and-the-power-of-learning-by-example.html)

The plan was then to write a script that would robotically perform those searches and extract the current URL from each result. But life’s short, I’m lazy, and I realized a couple of things. First, the desired result is usually but not always first, so the script would need to deal with that. Second, what if the URLs change yet again?

That led to an interesting conclusion: the search URLs themselves are good enough for my purposes. I just needed to transform the page of links to broken URLs into a page of links to title searches constrained to infoworld.com and my name. So that’s what I did, it works nicely, and the page is future-proofed against future URL breakage.

I could have written code to do that transformation, but I’d rather not. Also, contrary to a popular belief, I don’t think everyone can or should learn to write code. There are other ways to accomplish a task like this, ways that are easier for me and — more importantly — accessible to non-programmers. I alluded to one of them in A web of agreements and disagreements, which shows how to translate from one wiki format to another just by recording and using a macro in a text editing program. I used that same strategy in this case.

Of course recording a macro is a kind of coding. It’s tricky to get it to do what you intend. So here’s a related strategy: divide a complex transformation into a series of simpler steps. Here are the steps I used to fix the listing page.

Step 1: Remove the old URLs

The old URLS are useless clutter at this point, so just get rid of them.

old: <p><a href=”http://www.infoworld.com/article/06/11/15/47OPstrategic_1.html”>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

new: <p><a href=””>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

how: Search for href=”, mark the spot, search for “>, delete the highlighted selection between the two search targets, go to the next line.

Step 2: Add query templates

We’ve already seen the pattern we need: site:infoworld.com “jon udell” “TITLE”. Now we’ll replace the empty URLs with URLs that include the pattern. To create the template, search Google or Bing for the pattern. (I used Bing but you can use Google the same way.) You’ll see some funny things in the URLs they produce, things like %3A and %22. These are alternate ways of representing the equals sign and the double quote. They make things harder to read, but you need them to preserve the integrity of the URL. Copy this URL from the browser’s location window to the clipboard.

old: <p><a href=””>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

new: <p><a href=”http://www.bing.com/search?q=site%3Ainfoworld.com+%22jon+udell%22+%22%5BTITLE%5D%22″>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

how: Copy the template URL to the clipboard. Then for each line, search for href=””, put the cursor after the first double quote, paste, and go to the next line.

Step 3: Replace [TITLE] in each template with the actual title

old: <p><a href=”http://www.bing.com/search?q=site%3Ainfoworld.com+%22jon+udell%22+%22%5BTITLE%5D%22″>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

new: <p><a href=”http://www.bing.com/search?q=site%3Ainfoworld.com+%22jon+udell%22+%22XQuery and the power of learning by example%22″>XQuery and the power of learning by example | Column | 2006-11-15</a></p>

how: For each line, search for >”, mark the spot, search for |, paste, copy the highlighted section between the two search targets, search for [TITLE], put the cursor at [, delete the next 7 characters, paste from the clipboard.

Now that I’ve written all this down, I’ll admit it looks daunting, and doesn’t really qualify as a “no coding required” solution. It is a kind of coding, to be sure. But this kind of coding doesn’t involve a programming language. Instead you work out how to do things interactively, and then capture and replay those interactions.

I’ll also admit that, even though word processors like Microsoft Word and LibreOffice can do capture and replay, you’ll be hard pressed to pull off a transformation like this using those tools. They’re not set up to do incremental search, or switch between searching and editing while recording. So I didn’t use a word processor, I used a programmer’s text editor. Mine’s an ancient one from Lugaru Software, there are many others, all of which will be familiar only to programmers. Which, of course, defeats my argument for accessibility. If you are not a programmer, you are not going to want to acquire and use a tool made for programmers.

So I’m left with a question. Are there tools — preferably online tools — that make this kind of text transformation widely available? If not, there’s an opportunity to create one. What IFTTT is doing for manual integration of web services is something that could also be done for manual transformation of text. If you watch over an office worker’s shoulder for any length of time, you’ll see that kind of manual transformation happening. It’s a colossal waste of time (and source of error). I could have spent hours reformatting that listing page. Instead it took me a few minutes. In the time I saved I documented how to do it. I wasn’t able to give you a reusable and modifiable online recipe, but that’s doable and would be a wonderful thing to enable.

Why shouting won’t help you talk to a person with hearing loss

I’ve written a few posts [1, 2] about my mom’s use of a reading machine to compensate for macular degeneration, and I made a video that shows the optimal strategy for using the machine. We’re past the point where she can get any benefit from the gadget, though. She needs such extreme magnification that it’s just not worth it any more.

So she’s more dependent than ever on her hearing. Sadly her hearing loss is nearly as profound as her vision loss, and hearing aids can’t compensate as well as we wish. She’s still getting good mileage out of audiobooks, and also podcasts which she listens to on MP3 players that I load up and send her. The clear and well-modulated voice of a single speaker, delivered through headphones that block out other sound, works well for her. But in real-world situations there are often several voices, not clear or well-modulated, coming from different parts of the room and competing with other ambient noise. She depends on hearing aids but as good as they’ve gotten, they can’t yet isolate and clarify those kinds of voices.

One of the best ways to communicate with my mom is to speak to her on the phone. That puts the voice directly in her ear while the phone blocks other sounds. And here’s a pro tip I got from the audiologist I visited today. If she removes the opposite hearing aid, she’ll cut down on ambient noise in the non-conversational ear.

In person, the same principle applies. Put the voice right into her ear. If I lean in and speak directly into her ear, I can speak in a normal voice and she can understand me pretty well. It’s been hard to get others to understand and apply that principle, though. People tend to shout from across the room or even from a few feet away. Those sounds don’t come through as clearly as sounds delivered much more softly directly into the ear. And shouting just amps up the stress in the room, which nobody needs.

Lately, though, the voice-in-the-ear strategy — whether on the phone or in person — had been failing us. We had thought maybe the hearing aids needed be cleaned, but that wasn’t the problem. She’s been accidentally turning down the volume! There’s a button on each hearing aid that you tap to cycle through the volume settings. I don’t think mom understood that, and I know she can’t sense if she touches the button while reseating the device with her finger. To compound the problem, the button’s action defaults to volume reduction. If it went the other way she might have been more likely to notice an accidental change. But really, given that she’s also losing dexterity, the volume control is just a useless affordance for her.

Today’s visit to the audiologist nailed the problem. When he hooked the hearing aids up to his computer and read their logs(!), we could see they’d often been running at reduced volume. On her last visit he’d set them to boot up at a level we’ll call 3 on a scale of 1 to 5. That’s the level he’d determined was best for her. He’d already had an inkling of what could go wrong, because on that visit he’d disabled the button on the left hearing aid. Now both are disabled, and the setting will stick to 3 unless we need to raise it permanently.

Solving that problem will help matters, but hearing aids can only do so much. The audiologist’s digital toolkit includes a simulator that enabled us to hear a pre-recorded sample voice the way my mom hears it. That was rather shocking. The unaltered voice was loud and clear. Then he activated mom’s profile, and the voice faded so low I thought it was gone completely. I had to put my ear right next to the computer’s speaker to hear it at all, and then it was only a low murmur. When there aren’t many hair cells doing their job in the inner ear, it takes a lot of energy to activate the few that still work, and it’s hard apply that energy with finesse.

I’m sure we’ll find ways to compensate more effectively. That won’t happen soon enough for my mom, though. I wonder if the audiologist’s simulator might play a useful role in the meantime. When we speak to a person with major hearing loss we don’t get any feedback about how we’re being heard. It’s easy to imagine a device that would record a range of speech samples, from shouting at a microphone from across the room to shouting at it from a few feet away to speaking softly directly into it. Then the gadget would play those sounds back two ways: first unaltered, then filtered through the listener’s hearing-loss profile. Maybe that would help people realize that shouting doesn’t help, but proper positioning does.

Alternative sources of data on police homicides

There were empty seats at the table on Thursday for young males of color who have been shot by police, most recently Tamar Rice, a 12-year-old boy who was carrying a toy gun. His case resonates powerfully in Santa Rosa where, last year, 13-year-old Andy Lopez was shot for the same reason. He is memorialized in this moveable mural currently on display at the Peace and Justice Center around the corner from Luann’s studio:

Our son is an Airsoft enthusiast, just like  Tamar Rice and Andy Lopez were. Unlike them he is white. In circumstances like theirs, would that have made the crucial difference? We think so. But when you look for data to confirm or reject that intuition, it’s thin and unreliable.

Criminal justice experts note that, while the federal government and national research groups keep scads of data and statistics— on topics ranging from how many people were victims of unprovoked shark attacks (53 in 2013) to the number of hogs and pigs living on farms in the U.S. (upwards of 64,000,000 according to 2010 numbers) — there is no reliable national data on how many people are shot by police officers each year.

- How many police shootings a year? No one knows, Washington Post, 09/08/2014

The one available and widely-reported statistic is that, in recent years, there have been about 400 justifiable police homicides annually. In Nobody Knows How Many Americans The Police Kill Each Year, FiveThirtyEight’s Reuben Fischer-Baum reviews several sources for that number, and concludes that while it’s a reasonable baseline, the real number is likely higher.

Fischer-Baum’s article, and others he cites, draw on a couple of key Bureau of Justice reports. They are not hard to find:

https://www.google.com/?q=site:bjs.gov+”Justifiable+Homicide+by+Police

One report, Policing and Homicide, 1976-98: Justifiable Homicide by Police, Police Officers Murdered by Felons [1999], says this about race:

A growing percentage of felons killed by police are white, and a declining percentage are black (figure 4).

Race of felons killed
1978 50% White 49% Black
1988 59% White 39% Black
1998 62% White 35% Black

Felons justifiably killed by police represent a tiny fraction of the total population. Of the 183 million whites in 1998, police killed 225; of the 27 million blacks, police killed 127. While the rate (per million population) at which blacks were killed by police in 1998 was about 4 times that of whites (the figure below and figure 5), the difference used to be much wider: the black rate in 1978 was 8 times the white rate.

A more recent report, Homicide Trends in the United States, 1980-2008, is one key source for the widely-cited number of 400 justifiable police homicides per year:

Interestingly, the gap between justifiable police homicides and justifiable citizen homicides has widened in recent decades.

Table 14 addresses race:

It’s a complicated comparison involving the races of shooters and shootees when the shooters are civilians and also when they are police. In the latter case, the trend noted in the earlier report — “a declining percentage [of ‘felons’ killed by police] are black” — has reversed. Combining this table with the previous one, we get:

% of blacks killed by police
1978 49%
1988 39%
1998 35%
2008 38%

What is our level of confidence in this data? Low. From How many police shootings a year? No one knows.:

“What’s there is crappy data,” said David A. Klinger, a former police officer and criminal justice professor at the University of Missouri who studies police use of force.

Several independent trackers, primarily journalists and academics who study criminal justice, insist the accurate number of people shot and killed by police officers each year is consistently upwards of 1,000 each year.

“The FBI’s justifiable homicides and the estimates from (arrest-related deaths) both have significant limitations in terms of coverage and reliability that are primarily due to agency participation and measurement issues,” said Michael Planty, one of the Justice Department’s chief statisticians, in an email.

Are there other sources we might use? Well, yes. Wikipedia is one place to start. It has lists of killings by law enforcement officers in the U.S.. The introductory page says:

Listed below are lists of people killed by nonmilitary law enforcement officers, whether in the line of duty or not, and regardless of reason or method. Inclusion in the lists implies neither wrongdoing nor justification on the part of the person killed or the officer involved. The listing merely documents the occurrence of a death.

The lists below are incomplete, as the annual average number of justifiable homicides alone is estimated to be near 400.

Each entry cites a source, typically a newspaper report. About once a day (or maybe about twice day), a police officer shoots a civilian somewhere in the U.S. That’s a rare and dramatic event that will almost certainly be noted in a local newspaper. The report may or may not provide complete details, but it’s an independent data point. Everything else we know about the phenomenon is based on self-reporting by law enforcement.

To an analyst, the data that lives in Wikipedia tables is semi-structured. It can be extracted into a spreadsheet or a database, but extracting fully structured data almost always requires some massaging. The script I wrote to massage Wikipedia’s lists of police homicides handles the following irregularities:

  1. From 2009 to 2009 the table lives in a single per-year page. From 2010 onward, the per-year pages subdivide into per-month pages.
  2. The city and state are usually written like this: Florida (Jacksonville). But for the first five months of 2012 they are written like this: Jacksonville, Florida.
  3. The city name, and or the city/state combination, is sometimes written as plain text, and sometimes as a link to the corresponding Wikipedia page.
  4. The city name is sometimes omitted.

The script produces a CSV file. The data prior to 2009 is sparse, so I’ve omitted it. Here’s a yearly summary since 2009:

year  count
----  -----
2009     60
2010     82
2011    157
2012    580
2013    309
2014    472

It looks like this listmaking process didn’t really kick into high gear until 2012. Since then, though, it has produced a lot of data. And for one of those years, 2012, the count of police homicides is 580, versus the Uniform Crime Report’s 426. Each of those 580 incidents cites a source. Here’s one of them I’ve picked randomly:

One person is dead following an officer-involved shooting in Anderson, according to police.

Investigators said they were called to Kings Road about 2:45 a.m. Thursday for a domestic-related incident between a husband and wife.

Coroner Greg Shore said three officers went inside the home, and that’s when a man pointed what appeared to be a gun at officers. At least one officer fired his gun, and the suspect died on the scene, Shore said.

Investigators said the man’s wife, who was inside the home, was taken to the hospital due to injuries from a physical fight with the suspect. Shore said she was doing OK.

Shore said the man officers shot was 47-year-old Paul Leatherwood. Sgt. David Creamer said officers have been called to the same house six times in the past six months for domestic-related issues.

The State Law Enforcement Division is headed to the scene to investigate, which is standard protocol for officer-involved shootings in South Carolina.

- One killed in Anderson officer-involved shooting, FoxCarolina.com

We don’t know the race of the shooter or the shootee. We don’t know whether what appeared to be a gun turned out to be a gun. We don’t know whether this was or wasn’t reported as a justifiable homicide. But we could find out.

Every week, a million people listen to the blockbuster podcast Serial, an investigation into a cold case from 1999. A staggering amount of cognitive surplus has been invested in trying to figure out whether Adnan Syed did or did not murder Hae Min Lee. In this blog post, which I picked randomly from the flood of Reddit commentary, a lawyer named Susan Simpson has produced a 14000-word densely-illustrated “comparison of Adnan’s cell phone records to the witness statements provided by Adnan, Jay, Jenn, and Cathy.”

With a fraction of the investigative effort being poured into that one murder, we could know a lot more about many others.

The Church of One Tree: A civic parable

Juilliard Park is one of the jewels of Santa Rosa. It occupies 8.8 acres downtown, adjacent to the SOFA Arts District where, last weekend, thousands gathered for the 10th annual WinterBlast festival. Here’s how the Press Democrat describes the SOFA district:

Loosely gathered around the intersection of South A Street and Sebastopol Avenue, the neighborhood once had a shady reputation, but about a decade ago it began to change, and over those years it emerged as a destination for cuisine and culture.

And it continues to evolve. Today, you’ll find a picturesque cluster of small, independently owned shops, galleries, restaurants and even a live theater company.

We love the neighborhood and its energy. It was a major factor in our decision to move to Santa Rosa. When a small studio became available next door to the Atlas Coffee Company (labeled 1 on the map), Luann jumped at the opportunity. The timing was perfect. WinterBlast introduced hundreds of people to her work and to the stories that inspire it.

On the other side of the park is a landmark labeled Ripley Museum / Church of One Tree (2 on the map). Here’s the history:

The Church of One Tree was built in 1873 from a single redwood tree milled in Guerneville, California. The tree used to construct the Church stood 275 feet high and was 18 feet in diameter. Robert Ripley, a native of Santa Rosa, wrote about the Church of One Tree — where his mother attended services, — as one of his earliest installments of “Believe It or Not!” In 1970, Ripley repurposed the Church of One Tree as the Ripley Memorial Museum which was stocked with curiosities and “Believe it or Not!” memorabilia for nearly two decades. In 2009, the City of Santa Rosa restored the site adding several modern upgrades so that it could be utilized for every type of occasion.

Although that city web page doesn’t say so, the building was moved to Juilliard Park in 1957. It’s one of several landmarks that the city rents out for private events, so it’s no longer open to the public. The building is oddly sited. During the meeting Mayor Scott Bartley called it “backwards.” The front entrance faces the park, not the street, and is sheltered by a grove of redwood trees. That’s made it a magnet for the homeless who use the park. Private events have been disrupted; prospective renters have been spooked; a martial arts class that regularly rented the space found the situation untenable and bailed out.

To address these issues the city’s recreation and parks department proposed a fence that would enclose both the building and the nearby grove of redwoods. I’m not sure when or how I heard about the proposal (Luann hadn’t), but I attended this week’s city council meeting in part because the fence was on the agenda. I wanted to learn more about the issue, and to see how Santa Rosa would handle it.

Both the process and the outcome made me feel good about our new home town. Here’s the item that appeared on the council’s agenda for November 18:

12.2 REPORT – JUILLIARD PARK MASTER PLAN AMENDMENT CHURCH OF ONE TREE SITE ADDING A FENCE

BACKGROUND: The Recreation and Parks Department desires to increase use of the Church of One Tree and protect the building. The Church of One Tree site abuts Juilliard Park. The Master Plan for Juilliard Park was established in 1932, and was most recently amended on February 11, 2014. The Church of One Tree (Church) was placed on the site adjacent to Juilliard Park in 1957, and the building was used as the Ripley Memorial Museum from 1970 to 1998. The building was designated by City Council as a Landmark in 1998. The building has been restored and also modified to comply with the Americans with Disabilities Act.

A motion to recommend approval of the fence was approved by the Board of Community Services on July 23, 2014. A resolution to approve the fence design, with conditions, was approved by the Cultural Heritage Board on November 5, 2014.

RECOMMENDATION: It is recommended by the Department of Recreation and Parks that the Council, by resolution, approve the
Juilliard Park Master Plan Amendment, adding a fence to enclose the Church of One Tree.

When the item came up at the council meeting I thought it might be a done deal. Kelley Magnuson, deputy director of Recreation and Parks, opened her presentation with a slide summarizing the rationale for the fence:

  1. Increase use of the Church of One Tree
  2. Protect the building
  3. Incorporate redwoods within Church of One Tree

Slide 7 showed how the fence would reach into the park to enclose the redwood grove in front of the building. Slide 8 showed the view from Sonoma Avenue. Here, at the back of the building, the fence would block two entrances to the park. The gates (we later learned) would open only to admit guests to private events.

Although two bodies had endorsed the plan — the Cultural Heritage Board and the Board of Community Services — the council immediately began to ask about alternatives.

Councilor Erin Carlstrom:

Instead of building a fence, if we were to engage in more person-to-person contact — enforcement, security, interaction, funding homeless service providers…for example, what would it cost to increase our downtown bicycle patrol? Or adding security for events?

Councilor Ernesto Olivares:

I’m trying to understand how just having a fence solves the problem, it sounds like we have a bigger issue. If the drug use that was on the back porch of the church is now on the other side of the fence, we’re still dealing with an issue. What’s the broad plan to make the park — and the church — safer?

Kelley Magnuson was now in a tough spot. It was becoming clear that the problem she’d been tasked to address had been defined too narrowly. “Our objective today was to get your input on how to increase the use of the building, and protect it,” she said, “but I do agree there’s a larger issue.”

Mayor Scott Bartley, who is an architect, now made a not-entirely-facetious comment:

Why don’t we just pick the building up and turn it around so it faces Sonoma Avenue? Then we can just lock the door, it’ll look like a normal church, and nobody will think anything about it.

He then opened the public comment period.

First up was Ray Killion, who lives three doors down from the Church of One Tree. Here was the opening of his three-minute statement:

I’m against the direction of this fence. Aesthetically, a black iron fence is forbidding, it’s uninviting, it’s put there to say “you’re not welcome here,” and that’s not what I see as a proper message we want to send about our neighborhood.

Ray Killion made the following points, which were echoed by subsequent speakers:

  • Blocking both Sonoma Avenue entrances to the park would deny access to neighbors and visitors, as well as police, fire, and ambulance personnel.
  • The fence wouldn’t solve the crime and nuisance problems in the neighborhood.
  • If the fence must be built, at least keep the gates to the park open.
  • The Juilliard family had given the land to this city on the condition that “the whole of said property shall be forever used for park purposes only and for the use and benefit of the public in general and particularly the citizens of the city of Santa Rosa.” (This quotation from the deed was repeated several times during the evening.)

Referring to the language of the deed, Ray Killion concluded:

I would like to contrast that with what the parks department puts in the agenda tonight: “reserving this part of the park for the use of paid customers.” That’s not the purpose of a public park maintained with public money for the use of the public.

Jack Cabot has lived in the neighborhood for 24 years, owns 8 properties, and was deeply involved in the redevelopment of the SOFA district. In his statement he stressed how “eyes on the street,” which have multiplied thanks to the SOFA renaissance, would again diminish if the paths around the church were blocked.

Bob Wishard, another longtime resident, said that he and his wife had founded Juilliard Park’s original neighborhood watch 23 years ago. He added his voice to the “eyes on the street” chorus.

Floyd Fox reiterated the deed’s stipulation that the whole property was granted for public use. He added this quote: “a breach of any of the foregoing conditions shall cause said premises to revert to the said grantor, his heirs, or assigns.” He also cited resolution 23412 (1998), which established landmark status for the Church of One Tree. Quoting from the resolution — “the proposed Landmark has specific historical, cultural, and architectural value” — he concluded by asking: “Have you weighed the impact the fence would have on those values?”

Jim Macken extolled the park as a resource that should enhance the city’s ability to rent the building. Inappropriate uses of the park are “symptomatic of a larger problem” that the fence won’t fix, he said.

Edward Collins, a neighbor, opposed the fence because it would cut off access to the park and reduce citizen oversight. He also reiterated the deed’s stipulation of full public use. And he closed by referring to this clause in resolution 23412: “the Council found that the proposed Landmark designation is a Class 8 exemption under CEQA.” That exemption from the California Environmental Quality Act would, he argued, be jeopardized by the fence. He cited California Public Resources Code, Section 21084 and CEQA guideline 15300.2 in support of this argument. “If the city wants to proceed with the fence,” he said, “I think it will require a full CEQA review.”

(This was a nice civic moment. I don’t know what the councilors and city attorney were thinking but their faces said: uh oh.)

Next up was Jennifer Collins. “Being closed when not rented excludes the public from a cultural heritage landmark for the benefit of the paying few,” she said. “It punishes the neighborhood, not only by preventing us from using paths we all use regularly without incident, but also by sending a message to everyone that they are not welcome, and that this is a bad neighborhood.” She advocated for better lighting and for surveillance. And she argued that the city’s failure to maximize its revenue from the property is mainly a marketing failure. “There are no signs encouraging visitors to the Luther Burbank Gardens to come on over. Share a docent from there during peak tourist season to show off the church.”

Duane Dewitt, who often appears before the council, spoke next. “I’ve been going to this park since the 1950s,” he said. “When I was a boy we could sit under the redwoods on a hot day, and then go into the Ripley Museum.” He suggested using private security guards during events, and finding ways to open the building to the public at other times.

In her presentation, Lucinda Moore affirmed the historical value of the building, reiterated the importance of open access, and supported the idea of event security as an alternative to a fence.

Matt Martin, who is executive director for Social Advocates for Youth (SAY), was the next speaker. “The best practice for engaging with the homeless community,” he said, “is to do so face to face.” The Sonoma County Board of Supervisors recently allocated $925,000 for that purpose. In Santa Rosa, he said, it will fund outreach teams to engage with the homeless who live along the city’s creeks. He suggested that the city and SAY might be able to collaborate to bring such a team to Juilliard Park.

Next up was Cat Cvengros who is chair of the Board of Community Services. It was her board’s recommendation to build the fence. Now that idea was clearly in trouble. “When this item came before our board back in July,” she said, “we looked at the church as a revenue generator.” The fence does address the revenue problem, she said. “But you’re right, it does not address the larger issue.”

Anne Seeley, chair of Concerned Citizens for Santa Rosa, put her finger on the underlying issue. “We have at war here two different concepts. One is that a previous council directed Recreation and Parks to maximize income (unlike all other departments that aren’t required to) versus all the people who want keep the park open and free.”

That concluded the public comment period. Councilor Julie Combs now made a moving statement, part of which was quoted in the Press Democrat’s story (Santa Rosa council rejects fence at Juilliard Park) the next morning:

We are in some ways defining who we are as a community. We are making a decision about whether we put up fences and increase policing and security, or whether we fund park maintenance and alternatives for homelessness. If we fund park maintenance workers we put eyes on the park, we have a cleaner park, and we encourage people to attend. We have historically put our parks department in an untenable situation. We ask them to provide clean parks without providing them with alternative maintenance funds. I know that this council has turned down increased park maintenance funding on several occasions. So I ask staff to come back with a proposal for park maintenance.

The fence was now dead in the water. But since it was the active agenda item there needed to be a motion not to amend the park’s master plan to allow the fence. Some comments from discussion on that motion:

Councilor Robin Swinth:

As number of the neighbors pointed out, we’re dealing with a larger issue here. It’s an issue of homelessness, and it’s actually a regional issue. We need to get all the stakeholders at the table to resolve this. It’s the neighbors, it’s the homeless advocates, it’s the business owners, it’s the council, there needs to be a broader discussion.

Councilor Carlstrom:

I serve as our representative to the Russian River Watershed Association. One day a very excited woman came to us from the city of Oakland, extolling the virtues of a project they had implemented to install a new water filtration whiz-bang deal, and they’d gone through and cleaned out this big old homeless encampment. I looked at her and said: “Where’d they go?” She looked at me with a blank stare. I get it. You’ve got a siloed job. That’s what we’ve got here. I want to make sure we recognize my appointee to the Board of Community Services, Cat Svengros, as well as our Cultural Heritage Board, for their efforts on this. I know you took a lot of time to discuss this, and it was brought to you in a siloed way, and that’s your job. I want to be clear that I don’t overturn lower boards’ decisions lightly.

Mayor Bartley (echoing citizen comments about marketing the Church of One Tree):

We developed this building as a rental space. It was restored to be an income generator. The big issue — and it’s a different, more global issue — is how we do that. And I think it can be done. When I hear $350 to rent a church for three hours — that’s the bargain of the century. There should be more zeroes. We’re not marketing like we should. If we do that, and fill it with people, it’ll be a success.

Well done, Santa Rosa! Everyone involved was thoughtful, well-spoken, and open to compromise. Homelessness is a major issue here, and there’s plenty of frustration simmering, but the dominant tone wasn’t anger, it was compassion and a determination to work together to do the right things for the community as a whole. That’s part of what I came to see, and I wasn’t disappointed.

I also came to see how well the city’s online services support governance and citizen engagement. On that front there’s room for improvement. The video capture system works impressively well. You can find meetings — including the most recent one I attended on Tuesday — here. The service provider is Granicus, the same company that serves our former home town, Keene, as well as many other cities. It’s wonderful to be able to review council meetings online, anytime and anywhere. Back in 2008, in an interview with Tom Spengler, who was then CEO of Granicus, I was excited about the possibilities it opened up.

Soon after Keene implemented the Granicus service, though, I had to temper my enthusiasm. In Gov2.0 transparency: An enabler for collaborative sense-making I reflected on a key challenge: building accessible context around civic issues. Immediate stakeholders — government officials, citizens directly involved in decisions — create that context in meetings that are open, to be sure, but still often opaque to the uninitiated. Participants share a context that isn’t accessible to more casual observers.

Consider my situation. Our small investment in the SOFA district makes us minor stakeholders in issues affecting Juilliard Park. We’d like to be as well-informed about those issues as I am, now that I’ve plowed through hours of video and dozens of online documents. But that exercise was far too time-consuming to undertake on a regular basis, with respect to Juilliard Park or any other issue that affects us. And in fact, another such issue was on this week’s council agenda. We live in a neighborhood called the West End, near Railroad Square. There’s a train coming to town, and it runs right through our neighborhood. It’s a wonderful thing, and was in fact another factor in our decision to relocate here. But there’s always a tradeoff. In this case, it’s the possible closure of one of the streets in our neighborhood. Here’s a sign I pass every time I walk downtown:

It’s easy to joke about the URL for the draft environmental impact report, which is so long the sign can barely accommodate it. I’d rather my city’s content management system enabled it to form mnemonic URLs, like:

http:/srcity.org/SantaRosaRailroadCrossings

Which, of course, would also make a nice hashtag. In Tags for Democracy I showed how a city can promote a tag, like #SantaRosaRailroadCrossings, as a magnet for conversations that span multiple social networks and institutional websites.

But here I just want to focus on the page behind that formidable URL. It’s an overview of the project, with links to the draft environmental impact report as a whole (600+ pages!) and to the report’s individual sections.

The SMART train will stop at two stations in Santa Rosa, one of which doesn’t yet exist. Construction of the new Guerneville Road station will require a new railroad crossing at Jennings Avenue. Whether it should be an at-grade crossing or an elevated crossing is one key issue under discussion. A related issue is the possible closure of an existing crossing. An at-grade crossing at Jennings Avenue would be the simplest solution, but the California Public Utilities Commission rations the number of these. So adding a new at-grade crossing would require closing a street in our neighborhood. The elevated crossing wouldn’t entail that tradeoff. But as the visualization in the report shows, it’s a monstrosity.

I’ll bet few Santa Rosans have seen that illustration. Yes, the document is online, but it’s daunting. During upcoming conversations about the tradeoffs involved in choosing an at-grade or elevated crossing, wouldn’t it be nice to be able to link directly to that illustration?

Actually you can, and in fact I did just that two paragraphs above. Here’s the link behind the word visualization in that paragraph:

http://ci.santa-rosa.ca.us/doclib/Documents/CDP_Jennings_Avenue_DEIR_Aesthetics.pdf#page=23

It’s a little-known fact that you can form a link to any page within a web-hosted PDF file by appending #page=NUMBER to the URL. It’s challenging to get people onto the same page in open civic discussions; I wish this mechanism were more widely known and used.

Here’s another bit of information that could usefully be highlighted with a link. James Duncan lives near the Jennings site, and has crossed the tracks there for decades. In his statement to the council, he zeroed in on the state requirement to ration the number of crossings:

The pivotal, threshold issue — that isn’t really being discussed — is the position of the Public Utilities Commission to close a crossing in exchange for an at-grade crossing at Jennings.

It’s true there’s a general policy to maintain the status quo. And the interpretation, as I understand it, is that the crossing that exists at Jennings, and has been used all these years, is not [air quote] legal. But there’s no information about what constitutes legal. The federal government maintains an inventory of railroad crossings in the entire nation. But they have a category for what they call uninventoried crossings, and there’s a simple procedure for adding uninventoried crossings to the inventory.

Has James Duncan correctly identified a way out of the painful tradeoff at the heart of this issue? I don’t know, the council doesn’t know, James Duncan doesn’t know, but somebody knows. That person might be a government official or a private citizen (residing in Santa Rosa or elsewhere). A connection between that person and this issue might be brokered by a government official or by a private citizen. But one thing’s for sure. That person won’t want to wade through a 600-page report and hours of video. We’ll want to focus his or her attention on specific parts of documents, and specific parts of video testimony.

The Granicus service enables such linking. That’s how I was able to form the above link to James Duncan’s three-minute statement within the nearly 7 hours of video from Tuesday’s marathon session. But it’s cumbersome to create a link that jumps into the video at specific points. And using those links require a plugin (Flash or Silverlight), which rules out playback on most mobile devices.

If you do create a link, you’ll notice that the URL looks like this:

http://santa-rosa.granicus.com/MediaPlayer.php?clip_id=552&view_id=5&embed=1&entrytime=18150

In this example, entrytime=18150 denotes the number of seconds from the start of the video. It works out to 5 hours, 2 minutes, and 31 seconds, as you can see in this screenshot of the beginning of James Duncan’s statement:

Here’s what you see when you invoke the tool that helps you form a link:

The player pauses, and the clipping tool opens in a new window overlaid on top of it. Note that the beginning of the proposed clip doesn’t correspond to the 5:02:31 point at which the video is paused. (Click the image to enlarge it and see that more clearly.) You can scroll to that point within the clipping tool, but since the player is paused there’s no audio or video to guide you. To appreciate how clumsy that mechanism is, consider this screenshot from a Santa Rosa city council meeting that’s been posted to YouTube:

Right-clicking the video brings up a menu from which you can select Get video URL at current time. If you’re at the 1:20 mark in that video, the link you can copy and paste looks like this:

https://www.youtube.com/watch?v=zH9od5Tvuqg#t=80

That’s how simple and convenient it can be. And YouTube offers a further convenience. People navigate videos in terms of hours, minutes, and seconds. We’re not good at converting between that notation and raw numbers of seconds. But computers are really good at that. So YouTube supports this alternate syntax:

https://www.youtube.com/watch?v=zH9od5Tvuqg#t=1m20s

So really, you don’t even need a special clipping tool to link into a YouTube video at a specific point. You can just add minutes and seconds to the end of any YouTube URL. Their computers will figure out that 1m20s adds up to 80 seconds. Why should referring to a specific point in a city council meeting be any harder than that? It shouldn’t.

The programming to make deep linking in the Granicus player as convenient as deep linking in the YouTube player isn’t rocket science. Why hasn’t it been done? In my experience, these omissions happen because people don’t expect or demand capabilities that software could easily deliver.

Here’s something else that would expand access to archived council videos. Closed captions aren’t part of the service package that Granicus provides to every city, but Santa Rosa’s service includes them. If you turn on the closed captions while watching a Santa Rosa council video, you’ll see that they’re quite good — much better than the auto-generated captions available for YouTube videos. I suspect that’s true because Granicus provides a human transcriber as an optional part of its service.

Transcription quality notwithstanding, text synced to video is a powerful asset. In the Granicus implementation, it enables videos to be searched. For example, you can search the closed captions for Floyd Fox. Here’s the result:

The search returns three items because Mayor Bartley mentioned Floyd Fox three times: twice as an on-deck speaker, and then once as the current speaker. The third link jumps to Floyd Fox’s statement. Although you don’t land in quite the right spot — Floyd’s remarks begin at 3:06:40, the link based on caption search takes you to 3:07:00 — it’s amazing that you can search nearly 7 hours of video and quickly locate Floyd Fox’s statement.

But what if you didn’t know Floyd Fox was speaking? The names of citizens who make public comments don’t appear on the agenda, because they aren’t known in advance. During a meeting, people who wish to speak submit requests written on yellow cards. If the closed caption transcript were available alongside the video, you could scan within it to quickly absorb the sense of various parts of the meeting, and to find things that you didn’t know to look for. The transcript obviously exists. It can be displayed during video playback, and it can be searched. Why isn’t it available as a complete document? Again, it’s trivial for the software to do that. But nobody expects that feature, so nobody asks for it, and it doesn’t happen.

I first wrote about open government technology back in 2006, when Washington DC became the first city to publish data directly from its internal systems. In 2008 I explored a then-new service called Granicus. All along I’ve envisioned a world in which governments run transparently, publishing data that enables citizens and governments to work together. We’ve come a long way. But I am not yet satisfied. Even when meetings and supporting documents are available online, as they often now are, it’s harder than it should be to create the contexts needed for effective collaboration.

Context is, ultimately, a service that we provide to one another. If you’ve read this far, you know more about the fence around the Church of One Tree than anyone who didn’t attend the meeting. I created that context for you. Somebody else could do the same for the railroad crossing issue, and for any other issue in any other town. But it’s so painful to assemble that context that few will try, and fewer will succeed. Better tools aren’t the whole answer. Engaging with online civic proceedings isn’t everyone’s cup of tea. But if it were easier to do — fun, even — the motivated few could do powerful good for their communities.

The Nelson diaspora

This will be our first winter in California. I won’t miss New Hampshire’s snow and ice. But I’ll sure miss our regular Friday night gatherings with friends in Keene. And on Monday nights my thoughts will turn to the village of Nelson, eleven miles up the road. There, for longer than anyone knows, people have been playing fiddle tunes and celebrating a great contra dance tradition. On a cold winter night, when the whirling bodies of the dancers warm up the old town hall, it’s magical.

Gordon Peery, who for decades has accompanied the dancers on piano, once lent me a DVD documentary about the Nelson contra dance tradition. In a scene filmed at the Newport Folk Festival in the mid-1960s, the Nelson contra dancers appeared on the same stage as Bob Dylan and Joan Baez. I had no idea!

Here’s a video of a couple of minutes during a typical Monday night dance. To most people who don’t know the building, or the village, or the people, or the tunes, or the tradition that’s stayed vibrant there for so many years, it won’t mean much to you. To a few, though, it will resonate powerfully. That’s because Nelson, NH is the origin of a contra dance diaspora that spread across the country.

Although we aren’t contra dancers, we visited from time to time just to savor the experience. Then, a few years ago, in search of musical companionship, I began attending the jam that precedes the dance. There, beginning and intermediate musicians to learn how to play the dance tunes, mainly ones collected in these two books:

The Waltz Book opens with a tune written by Bob McQuillen, who played piano at the Nelson dance decades until his death in early 2014. And the book closes with a tune by Niel Gow, the Scottish fiddler who died in 1807.

The Waltz Book also includes a couple of Jay Ungar tunes, including Ashokan Farewell. Most people think it’s a tune from the Civil War. In fact Jay Ungar wrote it in 1982, and it became famous in 1990 as the theme of Ken Burns’ documentary about the Civil War.

The New England Fiddler’s Repertoire might also have included a mix of recent and traditional tunes. Instead it restricts itself to “established tunes” — some attributed to composers from the 1700s or 1800s, others anonymous. But it’s full of reminders that people have never stopped dancing to those traditional tunes. Here’s the footnote to Little Judique:

February 12. Played for a Forestry Meet dance in a barn with a sawdust floor at the University of New Hampshire in Durham. The temperature was 15 degrees below zero.

– Randy Miller, dance journal, 1978

As I page through these books now, and continue to learn to play the tunes in them, I’m grateful to have lived in a place where they were celebrated so well, and to have participated (in a small way) in that celebration. How will I continue that here? I don’t know yet, but I’m sure I’ll find a way.

Swimming against the stream

Congratulations to Contributoria! The Guardian’s experiment in crowd-funded collaborative journalism is a finalist in the digital innovation category of the British Journalism Awards. (Disclosure: Contributoria’s CEO and cofounder, Matt McAlister, is a former InfoWorld colleague.) The site, which launched in January 2014, runs on a 3-month cycle. So the November 2014 issue is online now, the December 2014 issue is in production, and the January 2015 issue is in planning. There are now ten issues archived on the back issues page. Here are the numbers from that page in a spreadsheet:

Who pays the writers? Contributoria’s business page explains:

The writers’ commissions are provided by the community membership pool and other sources of funding such as sponsorship. Initial backing came from the Google sponsored International Press Institute News Innovation Contest. We are currently funded by the Guardian Media Group.

Contributoria is a market with an internal currency denominated in points. I’m currently signed up for a basic membership which comes with 50 points I can direct toward story proposals. If I upgrade to paid membership I’ll have more points to spend. As a Supporter you get 150 points. As a Patron it’s 250 points plus delivery of each issue in print and e-pub formats. I like to support innovative experiments in journalism, such as the (dearly departed) Ann Arbor Chronicle [1, 2], so I may upgrade my membership. But I’m not sure I want to vote, with points, for individual proposals. I might rather donate my points to the project as a whole.

What I would like to do, in any case, is keep an eye on the flow of stories, cherrypick items of interest, and perhaps follow certain writers. So I looked for the RSS feeds that would enable me to do those things, and was more than a bit surprised not to find them. Here’s the scoop:

That was back in February. Nine months later Contributoria’s RSS feeds are still, presumably, climbing the todo list. How could a prominent and potentially award-winning experiment in online journalism regard RSS as an afterthought?

I mean no disrespect, and I won’t point any fingers because they’d point right back at me. I was an early adopter of RSS and an original member of the RSS Advisory Board. For many years an RSS reader was my information dashboard. I used it to organize and monitor items of interest from formal and informal sources, from publications and from peers. It was often the first window open on my computer in the morning.

And then things changed. I don’t remember exactly when, but for me it was even before the demise of Google Reader in July 2013. By then I’d already resigned myself to the notion that social streams were the new RSS. The world had moved on. Many people had never used RSS readers. For those who had, manual control of explicit lists of feeds now seemed more trouble than it was worth. Social graphs make those lists implicit. Our networks have become our filters. Mark Hadman and I might wish Contributoria offered RSS feeds but we’re in a tiny minority.

Of course the network-as-filter model isn’t new. The early blogosphere gave me my first taste of it. Back in 2002, in a short blog entry entitled Using people as filters, I wrote:

As individuals become both producers and consumers of RSS feeds, they can use one another as filters.

It worked well for years. I subscribed to a mix of primary sources and bloggers who were, in turn, subscribed to their own mixes of primary sources and subscribers. I often likened Dave Winer’s notion of triangulation — what happens when several sources converge on the same idea or event — to the summation of action potentials in the human nervous system. It was all very organic. I could easily tune my filter network to stay informed without feeling overwhelmed.

I doubt many of us feel that way now. Armando Alves certainly doesn’t. In Beyond filter failure: the downfall of RSS he laments the decreasing availability of RSS feeds:

I’m afraid the web/tech community has done a lousy job promoting RSS, and even people I consider tech savvy aren’t aware how to use RSS or how it would improve the way they consume information. Between a Facebook newsfeed shaped by commercial interests and a raw stream of information powered by RSS, I’d rather have the latter.

Let’s pause to consider an irony. Armando’s Medium page invites me to follow him there, but not by means of RSS. Instead the Follow link takes me to Medium’s account registration page. There I can log in, using Facebook or Twitter, then await the account verification email that will enable me to enter yet another walled garden.

There is, in fact, an RSS feed for Armando’s Medium posts. But only geeks will find it. Embedded in the page is this bit of code:

<link id=”feedLink” rel=”alternate” type=”application/rss+xml” title=”RSS” href=”/feed/@armandoalves”>

That tells me that I can subscribe to Armando in an RSS reader at this URL: https://medium.com/feed/@armandoalves

More generally it tells me that I can form the feed URL for any author on Medium by appending the @ username to https://medium.com/feed/.

How would a less technical person know these things? You wouldn’t. At one time, when RSS was in vogue, browsers would show you that a page had a corresponding RSS feed and help you add it to your feed reader. That’s no longer true, and not because the web/tech community failed to promote RSS. We promoted it like crazy for years. We got it baked into browsers. Then social media made it seem irrelevant.

But as Armando and many others are beginning to see, we’ve lost control of our filters. In the early blogosphere my social graph connected me to people who followed primary sources. Those sources were, it bears repeating, both formal and informal, both publications and peers. We made lists of sources for ourselves, we curated those lists for one another, and we were individually and collectively accountable for those choices.

Can we regain the control we lost? Do we even want to? If so let’s appreciate that the RSS ecosystem was (and, though weakened, still is) an open network powered by people who make explicit choices about flows of information. And let’s start exercising our choice-making muscles. I’m flexing mine again. The path of least resistance hasn’t worked for me, so my vacation from RSS is over. I want unfiltered access to the publications and people that matter most to me, I want them to be my best filters, and I’m available to return the favor. I may be swimming against the stream but I don’t care. I need the exercise.