SharePoint, IronPython, and another lesson in the virtue of laziness

I’m doing an internal project that involves reading several different data sources from a SharePoint 2007 server, merging them, and posting the merged data back to the server. Being lazy, I wanted to use IronPython, write as little code as possible, and do everything dynamically.

Reading the data sources, which are customized SharePoint lists (i.e., database tables), was straightforward. Every SharePoint list offers an “Export to Spreadsheet” link which produces an XML dump. Given that export URL, here’s a recipe for reading the data (from a Windows client that’s already authenticated to the server) and converting it to a list of Python dictionaries.

import clr
clr.AddReferenceByPartialName('System.Xml')
import System.Xml
from System.Xml import *
from System.Net import WebRequest
from System.IO import StreamReader

def getDataAsListOfXmlNodes(URL):
  request = WebRequest.Create(URL)
  request.Method = "GET"
  request.UseDefaultCredentials = True
  response = request.GetResponse()
  result = StreamReader(response.GetResponseStream()).ReadToEnd()
  doc = XmlDocument()
  doc.LoadXml(result)
  nsmgr = XmlNamespaceManager(doc.NameTable)
  nsmgr.AddNamespace( 'z', '#RowsetSchema')
  nodes = doc.SelectNodes ("//z:row", nsmgr )
  return nodes

def convertNodesToDicts(nodes):
  listOfDicts = []
  for node in nodes:
    attrs = node.Attributes
    dict = {}
    for a in attrs:
      dict[a.Name] = a.Value
    listOfDicts.append(dict)
  return listOfDicts

nodes = getDataAsListOfXmlNodes('http://host/sites/mysite/_vti_bin/...')
dicts = convertNodesToDicts(nodes)

Uploading my merged file to a document library on the server wasn’t so straightforward. I knew that SharePoint provides a set of web services APIs, so I started by acquiring the IronPython “Dynamic Web Services Helpers” from the Web Services sample. Among other things, these wrappers make it trivial to consume a WSDL-based web service. Here, for example, is a snippet that uploads a photo using the Imaging web service:

import System, clr
clr.AddReference("DynamicWebServiceHelpers.dll")
from DynamicWebServiceHelpers import *

filename = 'jon.jpg'
ws = WebService.Load('http://HOST/_vti_bin/Imaging.asmx')
ws.UseDefaultCredentials = True
bytes = open(filename,'rb').read()
bytes = map (ord, list(bytes))
bytes = System.Array.CreateArray(System.Byte,bytes)
ws.Upload('Photos','',bytes,filename,True)

So far, so good. But when I then looked for a generic service to upload any file to any document library, I found myself on a slippery slope. In the midst of exploring how to use the Swiss-Army-knife Lists service to accomplish a simple file upload, I realized I was working way too hard. Back in 2004, Bill Simser reached the same conclusion:

There seemed to be a lot of argument about using Web Services, lists, and all that just to upload a document. It can’t be that hard.

And it isn’t. As others have discovered too, SharePoint responds to a plain old HTTP PUT. Here’s an IronPython update to Bill’s recipe:

def upload(HOST,fname,rdir,rfile):
  wc = WebClient()
  wc.UseDefaultCredentials = True
  bytes = open(fname,'rb').read()
  bytes = map(ord,list(bytes))
  bytes = System.Array.CreateArray(System.Byte,bytes)
  url = '%s/%s/%s' % (HOST, rdir, rfile)
  wc.UploadData(url,'PUT',bytes)

And presto. A local file called, say, myfile.html, lands someplace like http://host/sites/mysite/MyLibrary/myfile.html.

Sheesh. If it feels like you’re working too hard, maybe you are. Step back, take a deep breath, and look for a lazier solution.