My longtime correspondent Raymond Yee, who I finally got to meet when I visited Berkeley last year, is writing a book on remixing data and services. The book mentions my elmcity.info experiment but, when Raymond visited the site the other day, all he saw was the text of the FastCGI script that runs the whole show. It turns out that when BlueHost upgraded Apache, they broke the mechanism that’s used to invoke arbitrary FastCGI scripts like my Django launcher.
It’s fixed now1, but the incident reminds me that I haven’t yet fully developed the line of thinking that I’ve now tagged servicemanagement. What I am increasingly feeling, as I’m sure many others are — and not only geeks, but also and especially civilians — is that it’s becoming impossible to sanely manage and control our growing numbers of service relationships.
That lack of control was the real point of my Verizon story, for example. And my story is tame compared to this one from John Halamka, which begins:
On Thursday, December 20, my FiOS internet/TV service was shut off by Verizon without any notice or warning.
and concludes:
As CIO of Harvard Medical School and CareGroup, I spend millions every year with Verizon and I cannot navigate Verizon Customer Service.
One aspect of the service management console I envision here would be a common view of all the services I depend on, green/orange/red for healthy/sick/dead.
In the case of a website, it’s not enough to know that the box (or webserver) is alive, healthy means delivering the expected page. There are a million ways not to, and so in the past — although not in this case — I’ve used a cache-and-compare method to verify. It would be entirely feasible for a web hoster to wrap a user-friendly interface around that method, so that anybody could easily declare expected outputs, but I haven’t seen this done by a commercial hoster.
Of course every service relationship sets up expectations, so there are all kinds of assertions you might want to make. I received the e-bill by the expected date. My paycheck was deposited. The payment I made was credited to the right account. The package I sent was delivered. The book I returned to the library was received and checked in.
I want a common way to make all these assertions, and to subscribe to notification of assertion failure.
I’d also like my personal service management console to be subscribed to streams of events, from service providers, about the operation of their services. The expiry notice from EZPass, the Apache migration email I may or may not have received from BlueHost, the WordPress upgrade notice that’s only visible when I log in to WordPress — every time I turn around, another of these alerts pops up, but they’re all coming at me in different contexts, and through different channels. It feels scattered and random because it is. But logically there’s just a handful of communication patterns — like event notification, assertion failure notification, and upgrade/renewal confirmation — that could be abstracted into a common interface. As services multiply, and as their management friction increases, the need will become more apparent.
1 Fixed for me, thanks to a specific intervention, but not, I observed and the BlueHost support guy confirmed, for others who wish to map arbitrary FastCGI scripts by declaring handlers in the control panel.
Fascinating idea. How would you like password management to work? Would you trust the console to keep your passwords? You could use a master password for them to encrypt all the others, but then you’d either have to (A) trust them with the master or (B) use the password every time the console checks your services.
I investigated “checking if your website is working” companies recently, though I haven’t seen a hoster offer it. Perhaps because it implies that there’s a chance that they might not have 100% full working uptime.
In general, until every upgrade is 100 percent backward compatible and everyone adheres to every standard (most of which do not currently exist) you will have to track down and adjust to changes completely out of your control and unknown to the activity causing the problem. It is the reason for years we did not apply Digital Equipment’s upgrades to their OS because each one would cause work which took more time than there was between upgrades. Sometimes it is amazing that the industry has gotten as far as it has. Only expectations of enormous gains in productivity and capability can drive such a system. Or maybe just a lot of people with too much time on their hands to play ;-)
“Would you trust the console to keep your passwords?”
Well I wouldn’t want to use passwords, I’d want to use information cards. But I don’t want to present one one a per-provider basis. So maybe as part of setting up a relationship with a provider I authorize it to write notifications into a cloud-based store that I control and where I access all providers’ data with a single use of my card.
“In general, until every upgrade is 100 percent backward compatible”
i.e. never
“you will have to track down and adjust to changes”
Agreed. It would be helpful, at least, to be able subscribe to streams of operational event data from your providers, at varying levels of verbosity, so you’d have a better idea of what changes are occurring.
Of course providers should tell you. It’s usually not automatic. But it could be. Might the sort of operational transparency that characterizes open source projects become a differentiator for some commercial services?
I think you are right that this is a real problem for millions of people and not just an early adopter issue. Have you looked at what Doc Searls and others are doing related to Vendor Relationship Management:
http://cyber.law.harvard.edu/projectvrm/Main_Page
I have been following it and there is certainly a huge problem to solve. It looks currently too broad. I am looking for that one neat must have feature that gets this idea off the ground.
We’ve thought about this is a future direction for the CoScripter project. What if you could subscribe to feeds that notify you of events of interest, such as a service going down, or a process getting stuck? Right now, if you’re lucky, you get an email notification from your bank that your check did not clear. If you’re not lucky, that state could pass unnoticed until months down the line when the credit company comes knocking at your door asking why you haven’t made your payment.
A few years ago I did some work on this topic, trying to model common processes such as the process of buying an item from Amazon. I was motivated by an incident that happened one Christmas: of the dozen or so gifts I ordered online, one never arrived, a fact which escaped my attention until February. By looking at the automatically-generated email from the vendor, and comparing the email flow from a healthy process versus a wedged process, a smart agent might be able to detect that something has gone wrong in the process and notify the user to take action. http://tlau.org/research/papers/kushmerick-iui05.pdf
One interesting problem is then how to gather all of your various notifications into a single common format so that they can be processed by such a system. For example, your bank notificataions may be only accessible by logging in to your bank website. Maybe we can use something like CoScripter to let end users define how to extract this information automatically for each of their services of interest.
Speaking of banks, I was stunned recently to learn that when I paid my mortgage (word derivation — gauge of death) with my bank’s online bill paying function, there really wasn’t much “online” about the actual transfer of funds. It turns out the mortgage company only accepts mailed checks, which I only realized when the process had hit a roadblock, causing a snippy call from my mortgage company. True this is a consumer example, but it’s even more important to avoid these issues at the business-to-business level because of the scope of consequences. A customer or a consumer works on the assumption that things function. If things don’t function, and there’s no notification, that relationship is in jeopardy. So I think your idea is clearly in the necessity category.
Interesting. My Django site on bluehost has suddenly stopped working. Now I know why.
I saw a hint on the BH forums that suggested changing/adding
AddHandler fcgid-script .fcgi
to .htaccess which fixed my Django app.