From screencasting to automation

I was pleased to see the announcement that Novell and Microsoft are collaborating on the User Interface Automation (UIA) stuff. My mom can use all the help she can get. But as I discussed in Automation and accessibility, beefing up our ability to automate software in a consistent way can give us huge leverage in other areas, like education, training, and collaboration.

In The social scripting continuum I suggested that a system like CoScripter could automate desktop and web applications in a common way. Here’s one way to think of the benefit of doing that. Today, I can share software-related task knowledge in a social manner by creating and posting screencasts. But you can only watch a screencast. If I could instead share that task knowledge in the form of standardized high-level scripts, you wouldn’t need to watch the screencast. Of course, you might want to, for other reasons, but not simply to get the procedural knowledge transferred from my brain and fingers to yours.

Given how popular screencasts have become in three years, I’ve got a hunch that taking things to that next level would be huge. And lord knows I’d love to be able to convey packages of procedural knowledge to my mom that way.

Posted in .

7 thoughts on “From screencasting to automation

  1. Agreed! Mom, click this link (link to script on my server). Optionally, you can pay attention. Then in one week when I remind you what we did (which you completely forgot), you can just run the script again.

    Then maybe can be measured on their ability to memorize procedural tasks by how few times they require watching the screencast before being able to complete it manually. :)

  2. Hey Jon. I may have missed some recent posts of yours on the topic, but have you seen the automated help in Office 12? Some help topics will literally show you how to do something. I was pretty impressed when I first encountered it.

  3. It’s 2015 and we still haven’t moved over to more automated screencasts. It’s a little depressing and I’m sure any screencast creators or eLearning publishing companies would make millions if they focused on automating the process.

    1. I thought about this recently after spending some time writing Selenium WebDriver scripts. In theory that’s one way to automate screencasts that you could then update as minor UX details change. In practice we’re not there yet, I don’t think, the browser isn’t inherently automateable and despite effort WebDriver is still quite trick to use.

      1. Indeed and it might be a little more possible with the browser than the desktop with cloud-based IDEs like Cloud9.

        I’m having more issues on the desktop especially on Windows where I have 4 different terminal windows open (for selenium, end2end testing, compiling code, etc.) and an editor and a browser.

        I would say my biggest issue is when you have to re-record things; how do you get your setup into state X-1 so that you can record it from state X to X+N?

Leave a Reply