In my Perspectives series I’ve been exploring some key relationships between Microsoft and external partners: Mercy Corps, NASA, the Library of Congress, the community of scientific and scholarly publishers, and individual scientific researchers including George Hripcsak and Dennis Baldocchi.

For this week’s show I spoke with Tony Hey, the VP of the External Research division of Microsoft Research. More than anyone else, he’s responsible for advancing what I’ll call the Jim Gray vision: All science is becoming computational and data-driven; a new breed of scientists requires a new generation of tools and services; Microsoft can and should rise to the challenge.

In coming weeks and months I’ll be reporting more of these stories of collaboration between Microsoft Research and scientific researchers. This is a treat for me because, little known fact, during the early part of my software career I took a hiatus to earn a graduate degree in science writing. I’ve always been curious about how science works, and this is an opportunity both to learn and to share what I learn.

It’s also an opportunity to expore, and to highlight, a side of Microsoft that you’ll never read about on TechMeme. When Tony Hey recently counted up the number of Microsoft Research engagements with university partners, the total exceeded 400. It’s an impressive outreach effort, and one that will benefit everyone: Microsoft, the scientists whose research work is enhanced by these projects, and all of us who depend on the outcomes of that research.

To prepare for my interview with Susan Gerhart I tried using text-to-speech software to read menu choices and text selections aloud. As always, I experienced the reaction that Susan, in her latest post, calls synthetic voice shock.

For those of us who don’t need to rely on synthetic voices, that reaction isn’t a problem, it’s merely a deterrent to optional use of the technology. For example, though it might be convenient to shift some material from the domain of written text to the domain of audio, the unpleasantness of synthetic voices stops me from doing that.

But the real problem, Susan explains today, is that synthetic voice shock deters people who have lost their vision, and who would benefit greatly if they could adapt to those voices. Here’s how she characterizes typical reactions:

• I cannot understand that voice!!!
• The voice is so inhuman, inexpressive, robotic, unpleasant!
• How could I possibly benefit from using anything that hard to listen to?
• If that’s how the blind read, I am definitely not ready to take that step.

She adds:

Conversely, those long experienced with screen readers and reading appliances may be surprised at these adverse reactions to the text-to-speech technology they listen to many hours a day.

How can we help people cross that chasm? Susan offers advice to four groups: “vision losers”, developers of assistive reading technologies, sighted people who are helping vision losers, and rehab trainers.

I’m in the third group. My mom’s macular degeneration is progressing, and although she’s not yet forced to rely on text-to-speech, that day may come. To those of us in this group, Susan recommends that, when evaluating applications and appliances, we need to bear in mind that voice quality is a separable concern, not directly tied to the capabilities of the software and hardware. And she suggests that, in order to help friends or family members, we might want to develop some familiarity with the range of available voices.

To that end, Susan has provided audio renderings of her blog posts, including four different versions of today’s post as read by Neospeech Kate, Neospeech Paul, Microsoft Mike, and Robotic UK Jane. None of the readings is pleasant to listen to. But Susan says:

I testify it takes a little patience and self-training and then you hear past these voices and your brain naturally absorbs the underlying content.

Those of us not compelled to learn how to “hear past” those voices might still want to try the experiment, in order to help friends and family members make the transition.

By an odd coincidence, my interview with Joshua Tauberer, founder and operator of GovTrack, went live just as the bill I’d been tracking using that system made headlines. Had I not been doing some investigation into legislative support for alternative biomass heating systems, I’d never have known that the Foreclosure Prevention Act of 2008 — which the President is now expected to sign into law this week, and which is generally known as the bailout for Fannie Mae and Freddie Mac — is also the bill that may result in tax credits for those of us in New England who’ve been investing in those heating systems.

This has been my most comprehensive attempt to open the lid of the legislative sausage factory, peer inside, and try to understand how a specific interest of mine was being processed. What I found is that, even with power tools like GovTrack and MAPLight, it’s really hard to make those connections. That’s partly because we lack good mechanisms to track the flow of bits of legislative language through an evolving assortment of bills, and to relate those fragments to the activities and interests of their sponsors.

But it’s also because a novice who tries to read and interpret this record lacks context. I had learned along the way, for example, that one of my senators cast a crucial NO on a cloture vote that would have amended the Energy Independence and Security Act of 2007 to include $22 billion of renewable energy tax incentives — including, as one small component, the “pellet stove” provision that has now landed in the Foreclosure Act. But I wasn’t at all clear on the significance, or recent evolution, of the parliamentary maneuver known as the cloture vote.

As Mike Caulfield points out here, there’s been a recent and dramatic increase in the use of this maneuver. As a result, “what your individual Senator does on bills matters quite a lot less than you think.”

The emerging breed of Congress-tracking tools and services can’t yet capture these nuances in ways that enable an ordinary citizen, motivated by some personal interest, to dive into the process and bootstrap an understanding that will guide voting or advocacy. Recognizing this, Joshua Tauberer explains in our interview that he’s currently exploring ways to crowdsource that bootstrapping effort. So for example, the site has a feature called “Users tracking this bill are also tracking…” Because I’m tracking both H.R. 6 (Energy Independence and Security) and H.R. 3221 (Foreclosure Prevention), the connection I made between these two seemingly unrelated bills creates a link for others to follow.

Mike writes:

As more people delve into these tools, the conversation around how our process really works can occur. And hopefully it can start to occur in channels outside the poliwonkablogosphere that I love so much but that others, well, not so much.

The first reaction to tools in the hands of novices is always that the novices don’t have the proper context. But use those tools and the context will come to you.

That statement resonates powerfully with two themes I’ve touched on before. One comes from John Willinsky, the educator, reading specialist, and open access advocate whom I encountered here, here, and here. There are only two factors that govern reading success, Willinsky says: motivation and context. To illustrate these principles, he talks about medical patients who, though they are nominally grade-8 readers, routinely surprise their doctors with grade-14 mastery of medical literature. How? When you or a loved one is sick, you’re intensely motivated to understand that literature. Yes, you initially lack context, but in the era of the Net, with the help of an ad-hoc community of others in the same situation, you can bootstrap that context.

The second resonance is with this Jeff Jonas meme: Data will find data, and relevance will find the user. I highlighted the social dimension of that idea in this essay: Data finds data, then people find people. Deepak Singh says these principles “define what the modern web is all about,” and adds that “without common data to commune around, there can be no communities.”

GovTrack is one of a number of efforts to create a framework of common data around which new political communities can form. I’ve also mentioned MAPLight, one of a number of projects funded by the Sunlight Foundation, whose chief data architect, Greg Elin, I interviewed here.

Admittedly these new services don’t yet close the gap that separates an ordinary citizen who’s motivated by a particular interest from a professional denizen of the poliwonkablogosphere. But it’s at least possible to leap across the gap, and we’re on the verge of building bridges across it.

In yesterday’s essay on my efforts to relate what our New Hampshire senators say about renewable energy to what they do or don’t do about it (and why), I left out a crucial step. It’s a nice illustration of the subtle and powerful data visualization provided by MAPLight.org.

I’d found references to a crucial NO vote cast by John Sununu, but was having trouble finding out for which bill, and for which step in its process, that vote was recorded. I thought the bill was H.R.6-110, and I was looking for a NO vote, cast by Sununu just before its Dec 13 passage, where the final tally was 59-40.

MAPLight’s use of a Timeline proved decisive. In this view of activity around H.R.6-110, scoped to Sununu, the circled region records two Sununu votes around that time. One red for NO, the other green for YES.

Mousing over the red NO vote got me very close to the answer:

This wasn’t quite right. Although the annotation shown in the screenshot reads Dec 07, the mouseover popup (which I couldn’t capture) says Dec 13, which turns out to be the date I was looking for.

It seems that this rendering of the timeline can’t accommodate the closely-spaced votes revealed in this details view:

But when I got to that view, and saw the 59-40 tally that had been reported, I knew I’d identified the crucial cloture vote.

Nicely done MAPLight!

While we’re on the subject, by the way, I wanted to pin down exactly what language was removed from HR 6 by the failure of that crucial vote. From a Congressional Research Service report entitled Energy Independence and Security Act of 2007: A Summary of Major Provisions, by Fred Sissine et al., I found this clue:

The White House threatened to veto the House-passed bill mainly because of the presence of provisions for an RPS [renewable portfolio standard] and for the repeal of oil and gas tax subsidies.

On December 7, 2007, a Senate cloture vote on the House-passed version of H.R. 6 — with provisions for an RPS and for the repeal of oil and gas subsidies — failed (52-43). After stripping out the RPS and modifying the tax package, a cloture vote on S.Amdt. 3841 failed (59-40).

To visualize what was subtracted from S.Amdt 3841 to produce S.Admt 3850, I dropped 3841 into Wikipedia, replaced it with 3580, and compared those two versions.

Here’s where the two part ways:

And here, deep in the thickets of 3841, is my beloved pellet stove provision in the left column, matched only by emptiness in the right column:

Now this surely isn’t an appropriate use of Wikipedia, so I’ve withdrawn the page I made to compare these two versions. But it’s odd, don’t you think? Why can we readily analyze the differences between two versions of a fanciful article about the heavy metal umlaut, but not so readily analyze the differences between two versions of a major piece of legislation?

Last week I spoke with Joshua Tauberer for an upcoming interview about his project, GovTrack.us, which is one of several efforts to make official sources of information about Congressional legislation available in more useful and actionable ways.

On Sunday, my local newspaper ran an article that sent me scrambling to GovTrack, to OpenCongress, to the Washington Post’s Congressional voting database, to the Library of Congress’ THOMAS, and to a flock of other sites. Why? The newspaper article, illustrated by a photo of NH Senator John Sununu standing with the owners of New England Wood Pellet in front of a pile of sawdust that will be pelletized, said:

Sununu supported a tax credit on high-efficiency biomass fuel systems, including wood-pellet stoves. The Senate passed the measure in April, in effect encouraging people to use the appliances, just as those who take on solar or wind power receive financial incentives for doing so.

I’ve invested rather heavily in clean wood-burning technologies — first in a pellet stove, now also in a wood gasification boiler — and I’m hoping these investments will qualify for tax credits. When phone calls to the local offices of my state representatives produced no results, I resolved to research the matter myself.

Here were my questions, along with the answers I found.

Q: Which bill approved by the Senate in April contains these incentives?

A: Bizarrely but typically, it’s not an energy bill. Instead it’s a housing bill, the Foreclosure Prevention Act of 2008, otherwise known as:

Building American Homeownership Act of 2008
Clean Energy Tax Stimulus Act of 2008
FHA Manufactured Housing Loan Modernization Act of 2008
FHA Modernization Act of 2008
Mortgage Disclosure Improvement Act of 2008
REIT Investment Diversification and Empowerment Act of 2008

Q: Which appliances will qualify?

A: The bill says:

(F) a stove which uses the burning of biomass fuel to heat a dwelling unit located in the United States and used as a residence by the taxpayer, or to heat water for use in such a dwelling unit, and which has a thermal efficiency rating of at least 75 percent.

On that basis I believe that both of my appliances should qualify.

Q: What is the status of that legislation?

A: According to GovTrack:

Status:

	Introduced	Jul 30, 2007
	Scheduled for Debate	Sep 5, 2007
	Amendments (235 proposed) [details]
	Passed House [details]	Aug 4, 2007
	Passed Senate	Apr 10, 2008
	Differences Resolved	–
	Signed by President	–

The bill may now proceed to
a conference committee of senators and representatives to work out
differences in the versions of the bill each chamber approved.
The bill then goes to the President before becoming law.
[Last Updated: Jul 20, 2008]

Cool! That’s what I needed to know. It ain’t law yet, but it might be. And thanks to a bevy of Web 2.0 tools and services, it was easy to find that out, right?

Wrong. It took me three hours on Sunday morning to sort this out. Along the way, I uncovered information that challenges the premise of the newspaper story. Entitled “Sununu touts alternative energy”, the story supports NH Senator John Sununu’s own claim to be a champion of long-term energy policy.

Here’s part of the analysis I posted in a comment to that newspaper article:

Sununu (and Gregg) were two of the votes responsible for the failure, by one vote, to include a $22 billion energy tax package in the Energy Independence and Security Act of 2007. Weirdly, that rejected tax package included Sununu’s original pellet stove provision from S1697!

So while the act became Public Law 110-140 in December, $22 billion of renewable energy incentives were jettisoned from it by Sununu. Why? The Congressional Quarterly noted that it “would have been offset in part by stripping about $13 billion in tax breaks from the oil and gas industry,” suggesting that Sununu was influenced by those interests.

Subsequently, I remembered Maplight.org where I found the breakdown for that crucial NO vote:

It’s utterly amazing, and deeply hopeful, that an ordinary citizen with no prior experience with an issue can dive in and do this kind of analysis.

And yet…I’m not really an ordinary citizen. It’s true that I’ve never before attempted this kind of detailed analysis. But I have an unusual degree of skill and perseverence when it comes to searching the Net, finding and using a variety of services, and correlating information from a variety of sources. At one point, trying to pull it all together, I had two dozen browser tabs open. That’s not normal.

It’ll take a few more turns of the crank before this kind of thing becomes routine enough for a truly ordinary citizen who cares about some issue to do the same kind of analysis.

On the first turn of the crank, we’ll need to get to the point where a Net-savvy individual can do what I did in minutes not hours.

On the second turn of the crank, we’ll need to deliver that capability into the hands of folks who aren’t freakishly Net-savvy.

I really believe that’s possible, maybe even inevitable. But challenges remain.

One of the biggest challenges, as Joshua Tauberer says in our upcoming interview and as my experience here confirms, is the analysis of the legislative texts themselves. Even with all of the excellent online services now available, it’s incredibly hard to relate colloquial discussion of legislation to the actual process and to related commentary. Every day, we read statements like this:

The Senate passed the measure in April, in effect encouraging people to use the appliances, just as those who take on solar or wind power receive financial incentives for doing so.

It took me hours to unravel that statement. For starters, it would help if newspapers would cite the legislation they mention. But that’s only a start. In this case, as is typical, the Sununu pellet stove provision began its life in one Senate energy bill that died, migrated into another Senate energy bill that died, and eventually wound up as part of a housing bill that passed. Separately, the provision was included in, then struck from, yet another energy bill that passed.

Crowdsourcing will be part of the answer. But to empower the crowds, we’re going to need power tools that help us visualize how specific bits of legislative language flow through the bewildering thicket of bills, to tie those bits of language to their sponsors, and to relate them to journalistic/blogospheric commentary.

Happily, this is just the sort of problem that should appeal to software engineers. The management of source code, like the management of legislation, is a high-stakes game. And in the software realm, we play that game really well. We take for granted the ability to reliably track changes in a large and evolving corpus of interrelated texts. When that same expertise is applied in the legislative realm, the results could be dramatic.

On a related note, I enjoyed this snippet from a Wired interview with Martin Wattenberg, whom I also interviewed here.

Wired:
Why is a numbers guy like you so interested in large textual data sets?

Martin Wattenberg:
Language is one of the best data-compression mechanisms we have. The information contained in literature, or even email, encodes our identity as human beings. The entire literary canon may be smaller than what comes out of particle accelerators or models of the human brain, but the meaning coded into words can’t be measured in bytes. It’s deeply compressed. Twelve words from Voltaire can hold a lifetime of experience.

Similarly, snippets of legislation can be the tickets to a more participatory democracy — if we can unravel and expose the contexts surrounding those snippets. It should be doable.

Although I’ve conversed online with John Faughnan since my days at BYTE, we’ve never met, and we had not even spoken on the phone until last week when he joined me on an episode of my Interviews with Innovators podcast. It was a great pleasure to finally connect in realtime with the prolific author of thoughtful analysis and commentary on things in general, on information technology, and on resources for parents of children with cognitive or emotional-behavior disabilities.

John was a country doctor, and he retains his medical license, but he doesn’t see patients nowadays. Instead he directs the development of clinical productivity software, with particular focus on methods of knowledge representation, and on strategies for effective collaboration.

We share a passion for strategies that entail simple but often overlooked uses of common software applications. For example, did you know that it’s possible, in Outlook, to edit the subject of an email message after it’s been received, and is just sitting in your archive? Try it, and you’ll find that you can. Color me amazed. I’m just the sort of personal information management geek who’d have discovered a hack like that, but I never did.

Now, why would you want to do such a thing? It’s a defensive strategy. The message entitled “Re: Next week” probably ought to be entitled something like “Consensus reached among A, B, and C on issue X for project Y.” Which title would you rather scan, in search results, six months later?

(John would like to find, and personally thank, the developer responsible for this feature, so if you know that person, or are that person, speak up!)

You can think of this technique as a kind of enhanced tagging. It’s related to a strategy for enriching email — embodying the journalistic principle of “heads, decks, and leads” — which I described in my book and in this report.

People mainly still think of information architecture as a discipline practiced only by designers and publishers. But what John and I have always thought is that we’re all becoming designers and publishers of streams of information, that those streams can all be navigated and searched in one way or another, and that the value of those streams depends on the ability of ourselves and others to navigate and search them effectively.

We also think that effectiveness requires two things. First, obviously, software that embodies the right principles and enables the right practices. But second, a broad awareness of right principles and practices. Those, we agree in this conversation, are not necessarily intuited by Gen X, Y, or Z just because they’re so-called digital natives. This stuff needs to be articulated, and it needs to be taught.

In the last installment of my little series on turning Internet feeds into TV feeds, I had decided to use IronPython to fetch data from the Internet, but C# to drive the WPF (Windows Presentation Foundation) application whose display my local public access TV station will broadcast. This division of labor between C# and IronPython arose because the XAML that drives the display needs to be refreshed periodically, and I didn’t know how, in IronPython, to properly delegate a timer-based event handler for WPF.

In this comment, Michael Foord, author of IronPython in Action and a major contributor to the IronPython Cookbook, showed me the way. Thanks Michael!

Based on his example, I’ve rewritten the C# program shown here as the IronPython script shown below.

I haven’t yet decided which version to deploy, but I’m leaning toward the IronPython version. Not because it’s more concise. It isn’t, really. Nor because I feel any need to use the same language for both components of the solution — that is, the feed fetcher and the feed displayer. I don’t care about language uniformity for its own sake.

I am, however, thinking that the folks at the TV station may want to modify these programs themselves. They’re pretty simple, and there’s no reason they shouldn’t be able to tinker with them. From that perspective, code that can be modified with nothing more than a text editor will be more accessible than code which requires a compiler.

I’m reminded of my early days as a website operator, when I was always glad to discover that a third-party application was written in Perl, rather than in C. That meant I could, and sometimes did, tweak the application in ways that otherwise would have been difficult or even (lacking C source code) impossible.

The difference here, of course, is that all of the underlying machinery — XAML, WPF, and the entire .NET Framework — is exactly the same¹ when approached from a scripting language like IronPython or a compiled language like C#. This ability to use common infrastructure from different langages — and from very different kinds of languages — has always seemed like a big deal to me, and still does.

¹ The same, that is, modulo the kind of boundary-crossing issue that stumped me until Michael Foord pointed me to CallTarget0, the wrapper for creating a delegate in IronPython.

import clr

clr.AddReferenceByPartialName("PresentationCore")
clr.AddReferenceByPartialName("PresentationFramework")
clr.AddReferenceByPartialName("WindowsBase")
clr.AddReferenceByPartialName("IronPython")

from System import *
from System.Windows import *
from System.Windows.Markup import *
from System.Windows.Media import *
from System.Windows.Input import *
from System.Windows.Threading import *

from IronPython.Runtime.Calls import CallTarget0

def LoadXaml(filename):
  from System.IO import *
  from System.Windows.Markup import XamlReader
  f = FileStream(filename, FileMode.Open)
  try:
    element = XamlReader.Load(f)
  finally:
	f.Close()
  return element

class Scroller(Application):

  def tickhandler(self,sender,args):
    def update_xaml():
      self.window.Content = LoadXaml(self.xaml)
    self.timer.Dispatcher.Invoke(DispatcherPriority.Normal, 
      CallTarget0(update_xaml))

  def __init__(self):
    Application.__init__(self)
    self.xaml = "scroller.xaml"
    self.window = Window()
    self.window.Content = LoadXaml(self.xaml)
    self.window.WindowStyle = WindowStyle.None       # go fullscreen
    self.window.WindowState = WindowState.Maximized  # 
    self.window.Topmost = True                       #
    self.window.Cursor = Cursors.None                #
    self.window.Background = Brushes.Black           #
    self.window.Foreground = Brushes.White           #
    self.window.Show()
    self.timer = DispatcherTimer()
    self.timer.Interval = TimeSpan(0, 60, 0)         # refresh hourly
    self.timer.Tick += self.tickhandler              #
    self.timer.Start()                               #

Scroller().Run()

The Conversations Network is embarking on a new phase in which it will expand its ambition to capture, publish, and curate spoken-word audio from a wide range of sources. One of the challenges will be to help more people effectively capture audio to a reasonable standard of quality. Dan Bricklin, my guest for this week’s ITConversations show, has ascended that learning curve in recent years. In this conversation he explains why he’s become interested in audio recording, and what he has learned about equipment, and techniques, which can be readily transferred to individuals and organizations wanting to make decent recordings of their own events.

When I embarked on my personal audio adventure a few years ago, I naively thought that our fancy new digital technologies would make the whole process very simple. Boy, was I wrong about that. Yes, we’ve made digital photography accessible to the masses, but there was vast demand for enabling the so-called Happy Snapper to point, shoot, and take a decent photo. There’s been comparatively little demand for enabling the Happy Caster to plunk down a microphone, punch record, and capture a decent sound track.

Over the last few years I’ve slowly and painfully assimilated just a fraction of the audio lore possessed by domain experts like the Conversations Network’s founder Doug Kaye, and its senior audio engineer Paul Figgiani. So it was refreshing to hear from Dan Bricklin that it has also been a struggle for him to become competent in this domain.

I guess the demand for point-and-shoot photography will always outstrip, by orders of magnitude, the demand for plunk-and-punch audio recording. But the latter demand is growing, and in this conversation we speculate a bit on what the Happy Caster solution might be.

Mainly, though, Dan focuses on two things. First, the new opportunity to capture spoken-word events that would otherwise be lost, and publish them for audiences that didn’t attend, or couldn’t have attended, in person.

Second, the minimal setup that will enable folks who are not audio experts to accomplish that capture and publication.

PS: A bit of backstory on this recording illustrates some of the challenges of the audio domain. In my FAQ for interviewees, I invite remote interviewees to record themselves locally, then send me the track which I combine with my own locally-recorded track. Why? If you’re sending voice over the network, whether it’s POTS (plain old telephone service) or Skype, there’s a lot that can and often does go wrong. Eliminate the network and you avoid all those problems.

In principle, combining local tracks recorded separately is a great solution. In practice, it has almost never worked out, and this case was no exception.

Usually the problem is that interviewees lack the gear or knowledge required to make a decent local track. Attempts to record directly into a computer always end badly. Most people don’t own standalone digital audio recorders. In one case, a musician who routinely records his music through a mixer nevertheless produced an unusable track because he’s not used to recording his voice and overshot the limits.

In this case, Dan was quite capable of making a good recording, and he did, but things went wrong on my end. What Dan recorded was an MP3 file. What I was expecting was a WAV file, because I was going to edit the combined recording and it’s dicey to uncompress an MP3, edit, and then recompress.

Now, Dan had recorded the MP3 at a bit rate — 192kbps — that he judged would be high enough to survive an edit. But would our discriminating audio engineer Paul Figgiani agree? We weren’t sure, so I sent Paul samples of Dan’s MP3 track and the WAV file I made from the telephone track I’d recorded using the Telos. Paul’s verdict: “I think we can make the 192 kbps mp3 version work. The bit rate is high enough … lets go with it.”

So far, so good. But when I loaded up my local track and Dan’s remote track into Audition, things didn’t line up. My WAV track was slightly longer (or shorter, I can’t remember) than Dan’s MP3 track. The difference was only about 1.5 seconds over an hour-long recording, but still, it had to be dealt with.

Audition has a time-stretch feature that can be used to solve this problem. And I could swear that I’ve used it successfully before in these circumstances. But this time, I couldn’t make it work. Every time I tried to stretch the shorter clip, it snapped back to its original position. I fiddled with every approach I could think of, or could discover by searching, and finally threw up my hands and just used the original recording that had both halves of the conversation in sync. If this Audition behaviour rings a bell with anyone, I’d love to know what went wrong and how to avoid it next time.

The moral, anyway, is that if a reasonably technical guy like me is struggling to keep his head above water in this domain, it’s clear that non-geeky civilians will just drown. I’m quite curious to know when, or perhaps whether, those civilians will constitute a market that technology providers want to serve.

The folks at National Public Radio love to create driveway moments:

You’re driving along, listening to a story on NPR. Suddenly, you find yourself at your destination, so riveted to a piece that you sit in your idling car to hear it all the way through. That’s a Driveway Moment.

The podcasting counterpart, for me, is the Ashuelot Moment. I’m jogging along the Ashuelot River, and I’m so riveted to a piece that I take a longer route so my run won’t end before the story does.

The Long Now podcasts are my most reliable source of Moments but they’re only on a monthly cycle. TED talks are another good source, though I’ve lost track of how to subscribe to the comprehensive audio-only feed. The Conversations Network, to which I contribute a weekly show, produces occasional Moments, but a lot of the material there is so closely aligned with my own particular interests and inclinations that it doesn’t often surprise or challenge me.

Another good source is Christopher Lydon’s Open Source, which launched in 2005, suffered a setback in 2006, and then recovered in 2007. It took me a while to reconnect after the hiatus, but now I’m finding it to be more stimulating than ever.

Here’s my most recent Moment, from this Open Source show with Ethan Zuckerman and Solana Larsen. Ethan is speaking:

My hope was that with the Internet, suddenly we’re all connected, we hold hands and sing Kumbaya. And it just hasn’t worked out that way.

…
You loook at a site like Digg, or Reddit, these are sites that promised the future of journalism. We’d all get together and decide what’s important. But, who’s we? Or as per the Lone Ranger, who’s we, white man? Or more to the point, who’s me, white geek?

If you’re getting your news from these sites, you’re getting a very particular, tech-heavy view of politics, a fairly focused view of the world. And you start falling victim to homophily, which is what happens when all of your news and opinions are coming from people who’ve got the same background and the same values as you.

Homophily is the tendency of birds of a feather to flock together. It’s the tendency to walk into a room, find the person most similar to you, and form a bond. It’s a natural human tendency, but it’s probably worth fighting against. Homophily makes you stupid.

Of course I share tribal affiliations with Ethan Zuckerman, so I’d have been likely to find that particular show one way or another. But Global Voices Online, the project that Ethan and Solana discuss on that show, is all about resisting homophily, and enabling us to tune into global perspectives offered by people in circumstances very different from our own.

Just because we can, though, doesn’t mean we will. Homophily is a natural tendency. It’s easy and comfortable to immerse ourselves in the familiar. It’s hard and uncomfortable to seek out the unfamiliar. How do we overcome that?

Recommendation systems don’t help me much. They only suggest things similar to other things I’ve shown interest in. Increasingly that just frustrates me. The most delightful recommendations are those that connect me with things that interest me in unpredictable ways. That happens serendipitously, and I haven’t yet found a reliable way to manufacture the serendipity.

Lately I’ve started to wonder about the notion of anti-recommendation systems. One example of an anti-recommendation system is LibraryThing’s UnSuggester, which find books least likely to coincide with yours. It’s a whimsical feature that honestly hasn’t been useful to me yet, but I think the idea merits exploration and development.

Although it isn’t automated or automatable, I’d argue that the Passion Thursday series on Open Source is a kind of anti-recommender. The series includes shows about birdwatching, the pursuit of truth, poker, the potato, cursive handwriting, and the theremin, an early electronic instrument recently notable in the repertoire of the indie band DeVotchKa. The only common thread is someone’s passionate interest in something.

We’re not inclined to resist homophily and seek out otherness. But passionate storytellers can take us to places we wouldn’t otherwise go, and create Moments there.

Passion is a good way to lubricate the engine of serendipity.

In conversation with English and Welsh friends last week, the subject of Britain’s imminent National Identity Scheme came up. My friends, who are worldly and well-educated but not technical, voiced concerns about the amount of personal information that will be stored. Their understanding was that a lot of this information will be kept on the new ID card. In fact, the proposal says that only a subset will stored on the card, which will be backed by a cloud-based (and decentralized) National Identity Register. But either way, my friends’ concerns are of course valid. If governments or businesses aggregate too much personal information, accidents and abuses will occur.

At the same time, my friends do recognize the need for a strong and secure means of identification. So they’re not opposed to identity cards on principle, they just don’t want those cards to contain, or link to, extensive dossiers.

At this point, channeling Kim Cameron, I launched into an explanation of the laws of identity and the identity metasystem. Well, sort of. I didn’t say anything about cryptography, or digital certificates, or XML web services. But I did paint a picture of a world in which individuals interact with many identity providers and many relying parties, in which all actors trust one another in exactly the ways they already do today, and in which disclosure of personal information is minimal and context-dependent.

Halfway through I thought, well, this will never fly. This whole scheme is based on decentralization and indirection, and I know people don’t take naturally to those concepts.

But…they completely got it! Maybe that’s because the threat of a monolithic system leads people to appreciate the virtues of a decentralized one. Maybe it’s because ongoing experience with the Net makes people more comfortable with the principle of indirection. Maybe it’s both these factors and others as well. In any event, it was a hopeful moment. Identity geeks have struggled, for many years, not only to devise right systems, but also to motivate an understanding of what makes systems right, and why. Now that right systems are coming into existence, it’s good to see that (some) people are ready to appreciate and embrace them.

On my Perspectives show last week, Curtis Wong and Roy Gould relate the history and educational mission of the WorldWide Telescope. On this week’s show, principal developer Jonathan Fay describes how the underlying technologies enable the WWT’s seamless view of the sky.

There were a bunch of things I wanted to know, including:

How does the WWT project build on, and extend, the SkyServer project to which Jim Gray made fundamental contributions?

What standards and protocols enable the various sky surveys to be woven together?

What’s the relationship between Deep Zoom and the WWT’s own scheme for managing and viewing tiled multi-resolution imagery?

How much of the data is stored on Microsoft servers, how much is stored elsewhere, and in what ways do the supporting data services cooperate?

Jonathan answers all these questions, and he also answers one I didn’t think to ask:

What technique is used to project the stars onto an imaginary sphere at near-infinite distance?

The answer to that last question is that a new kind of spherical projection had to be invented:

Imagine taking a round room, and trying to put a bunch of bathroom tiles on it, and grout it. The tiles seem to come together and have parallel lines for a while, but eventually it stops working well. Maybe you can take one line around the equator, but as you go up you have fewer tiles, and weird-shaped tiles, and nothing lines up.

That’s the problem we have. We’re looking at spherical data, so we had to come up with a new spherical transform that preserves the poles. In previous projects, like Virtual Earth or TerraServer or Google Earth, the poles weren’t important, because nobody lives there and nobody needs map directions for driving around there.

So we had to come up with something called TOAST: tesselated octahedral adaptive subdivision transform. It creates a 360-degree wraparound view that’s either a planet surface or the infinite sphere of the sky, and lets you represent it using a 3D graphics accelerator, very rapidly and efficiently. So we can have an image pyramid the way Deep Zoom does, and TerraServer before it, but we don’t have to give up the poles.

This transform isn’t proprietary, and in fact it’s being applied to the 50-odd full-sky surveys hosted at NASA’s SkyView virtual observatory. The implications are pretty astounding. This imagery is stored in astronomical databases using what’s called tangential projection, which suffers from polar distortion when combined into large mosaics. Now the imagery can be combined into large mosaics — or indeed a complete view of the sky — and seen without distortion. What’s more, multiple surveys can be aligned to that spherical projection. That’s why, in WorldWide Telescope, you can cross-fade between a view of the Milky Way in visible light and views in infrared or ultraviolet light.

What the WorldWide Telescope really is, Jonathan says, is a browser, like a web browser but for an information space defined in astronomical terms. Here’s how he sums up the work that was necessary to make that possible:

The vision of getting everybody access to all this astronomy data required systematic changes at every single level. We built on some things that Jim pioneered with NVO, and worked from there, but it was very systematic. How people process the data. The client to access the data. The protocols over the wire. Educating people, providing the context for it.

We put a lot of things together, but we also created a systematic model for how to do everything end to end, top to bottom, left to right. Now there may be other people who use the pieces that we’ve created, and then change them to use different data sources, different visualizations. Say someone creates a Mac client, or an iPhone client, that’s possible. Or a mobile phone version of it, or a web-based version. Over time we or others can replace various components, but as a reference model for solving all the problems in order to get the data into people’s homes and into their eyeballs — you had to solve for all of those problems, otherwise people are still blocked from being able to really explore.

For Curtis Wong, the WWT is an extension of John Dobson’s sidewalk astromony — a way to bring telescopes to the public and to enable astronomers to share their knowledge of the sky with everybody. For Jonathan Fay, it’s the perfect application of earth and sky visualization technologies he’s been developing throughout his career. Their interests and talents combined, as Jonathan says, like peanut butter and chocolate:

Curtis had been exploring how to create an educational environment with rich tools for exploring space, and he’d been collaborating with Jim Gray on TerraServer, and now he was looking for the technology to make it possible.

Here I had this technology, and was looking for somebody who was enthusiastic about having a purpose for it. So it was the peanut butter and chocolate moment.

Yum.

Last week I started looking into ways to Internet feeds into TV feeds. Although I did come up with a way to turn a data feed into a video file, that wound up being overkill. It turns out that the local station is willing to broadcast the signal from a computer display. To create that signal, several folks suggested using PowerPoint, but I found that its scrolling credits feature doesn’t accommodate really long lists of credits. So I decided to try XAML, the application markup language that works with Silverlight and the Windows Presentation Foundation (WPF), in concert with IronPython.

The plan was as follows. A long-running IronPython script periodically fetches the feed from a web service, interpolates the text into a XAML template that animates the crawl, and displays the XAML in a fullscreen white-on-black WPF window.

Here’s the XAML template:

<Grid xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"  
  Width="%s" Height="%s">
<TextBlock xml:space="preserve" FontSize="%s" Margin="%s,%s,0,0" 
  FontFamily="Arial">
    <TextBlock.RenderTransform>
      <TranslateTransform x:Name="translate" />
    </TextBlock.RenderTransform>
  <TextBlock.Triggers>
    <EventTrigger RoutedEvent="FrameworkElement.Loaded">
      <BeginStoryboard>
        <Storyboard RepeatBehavior="Forever">
          <DoubleAnimation From="%s" To="-%s" 
            Storyboard.TargetName="translate" 
            Storyboard.TargetProperty="Y"
			Duration="00:%s:%s" />
        </Storyboard>
      </BeginStoryboard>
    </EventTrigger>
   </TextBlock.Triggers>
<Run>
<![CDATA[
%s
]]>
</Run>
</TextBlock>
</Grid>

Some of the values depend on the number of items in the feed, so the script interpolates those values into the template. Then it formats the feed and plugs the formatted text into the template’s CDATA section. The formatted text looks like this:

EVENTS FOR MON JUN 30 2008 FROM THE ELMCITY.INFO CALENDAR

06:00 AM:  lap swim (ymca)

07:00 AM:  AA: On Awakening Group (eventful: Keene Unitarian
      Universalist Church)

After generating the XAML, the IronPython script fires up an Application object, creates a window, loads in the XAML to start the crawl, and sets a timer to refresh the XAML.

I ran into a snag when I tried to set that timer, though. There are a few different timers you might imagine using in this context, including Python’s own timer object and various timers available in the .NET Framework. All but one of these, however, will complain about invalid cross-thread access when you try to update the application’s user interface from a timer event handler.

The right timer to use, it turns out, is .NET’s System.Windows.Threading.DispatcherTimer. But when I tried it, I ran into another snag. In C#, you create a WPF-friendly timer like so:

DispatcherTimer timer = new DispatcherTimer();
timer.Tick += new EventHandler(event_handler);

event_handler is a method, but EventHandler returns a delegate that encapsulates that method. I couldn’t find a straightforward way to create a delegate, and do that encapsulation, in IronPython.

If you know how, I’d love to hear about it. Then again, it really doesn’t matter. Logically this program has two loosely-coupled parts. The engine part reads the feed from a web service and formats it as XAML. It can be a Python script that runs on a scheduled basis to fetch and format the feed.

The user interface part loads, displays, and then periodically refreshes the XAML. It can be a little C# program that runs forever, displays the animation, and refreshes the data, like so:

using System;
using System.Windows;
using System.Windows.Markup;
using System.Windows.Input;
using System.Windows.Media;
using System.IO;
using System.Windows.Threading;

namespace CalendarCrawl
{
  public class CalendarCrawler
  {
    static private Application app = new Application();

    static private StreamReader getXaml()
    {
    StreamReader sr = new StreamReader("WPF.xaml");
    return sr;
    }

    [STAThread]
    static public void Main(string[] args)
    {
        Window win = new Window();
        win.WindowStyle = WindowStyle.None;           // go fullscreen
        win.WindowState = WindowState.Maximized;      // go fullscreen
        win.Topmost = true;                           // go fullscreen
        win.Cursor = Cursors.None;                    // go fullscreen
        win.Content = XamlReader.Load(getXaml().BaseStream); 
        win.Background = Brushes.Black;
        win.Foreground = Brushes.White;
        DispatcherTimer timer = new DispatcherTimer();
        timer.Interval = new TimeSpan(0, 1, 0);       // every hour
        timer.Tick += new EventHandler(eventHandler); // wire up handler
        timer.Start();
        app.Run(win);
    }

    static private void eventHandler(Object sender, EventArgs args) 
    {
        app.Windows[0].Content = XamlReader.Load((getXaml().BaseStream));
    }
  
  }
}

It was odd how reluctantly I came to this division of labor. Evidently I still need to remind myself that in a world of loosely-coupled applications and services, when you need to get something done, There Is More Than One Way To Do It.

Here’s another way. If the engine doesn’t have to talk to the .NET Framework’s WPF machinery, there’s no need to use IronPython. Any flavor of Python makes a handy tool for talking to RESTful web services, wrangling text, and interacting with the file system.

Here’s yet another way: A Silverlight version of the user interface. It’s nice to know that option is available. However, I’m leaning toward the C# version. The target machine is Vista, it already has .NET and WPF, why use a long-running browser instance just to host this tiny little thing?

One final point is worth mentioning. XAML is really just another source language for the .NET runtime and framework, like C# and IronPython and others. You can, for example, create an application window by writing a Window tag in XAML markup, and specifying parameters as attributes. Or you can do it by invoking System.Windows.Window from IronPython or C# or another .NET language, and specifying parameters in code. The boundary between markup and code is very fluid, and you can draw the line for reasons of convenience, maintainability, and taste. It’s a very flexible system, and it becomes even more flexible when you can use a dynamic language like Python to generate the XAML, the code, or both.