In this interview at MIX, Mark Rideout explains how Silverlight will use the same UIA (User Interface Automation) mechanisms that make Windows apps (and will make Linux applications) accessible by way of assistive technologies like screenreaders.
If you’re not somebody who needs that kind of assistance, you may not think this matters to you. But as I’ve pointed out in a series of essays, the flip side of accessibility is automation, and that’s something we all need.
For software developers, the automation framework provides the hooks needed to test the interactive behavior of applications.
For users, it provides the hooks needed to record, exchange, and replay software interactions. In The social scripting continuum I showed how IBM’s CoScripter enables people to share their knowledge of how to use web applications. It’s fabulous, but it’s restricted to the domain of simple web apps running in Firefox. IE, Ajax, Flash, Silverlight, and desktop apps are all out of scope.
With an automation/accessibility framework common to browsers, rich runtimes in browsers, and desktop apps, you could in theory enable a common way for people to describe and share their knowledge of how to use software across the full range of application types, for any browser, any rich runtime, and any operating system.
We’re not there yet, and we may never get there, but this Silverlight announcement points toward a future that’s worth imagining.
Update: In related news, John Resig notes that IE8 supports the W3C’s ARIA (Accessible Rich Internet Applications), which makes Ajax applications accessible to screenreaders. Here’s a brief guide for the perplexed, myself included, because this stuff is a layer cake.
Native accessibility toolkits, like MSAA (Microsoft Active Accessibility) and ATK (Linux Accessibility Toolkit), are the foundation. The Mozilla implementation of ARIA rests on this layer, as does the IE8 implementation. User Interface Automation (UIA), meanwhile, is part of the .NET Framework. It can be used to automate unmanaged apps like Word, as well as managed apps on the desktop or (now) in Silverlight. How UIA will be realized on Linux is something I don’t know, but would like to find out.
I can’t formulate a unified field theory that joins all these pieces, on various platforms, but I hope one will emerge.