SharePoint, IronPython, and another lesson in the virtue of laziness

I’m doing an internal project that involves reading several different data sources from a SharePoint 2007 server, merging them, and posting the merged data back to the server. Being lazy, I wanted to use IronPython, write as little code as possible, and do everything dynamically.

Reading the data sources, which are customized SharePoint lists (i.e., database tables), was straightforward. Every SharePoint list offers an “Export to Spreadsheet” link which produces an XML dump. Given that export URL, here’s a recipe for reading the data (from a Windows client that’s already authenticated to the server) and converting it to a list of Python dictionaries.

import clr
clr.AddReferenceByPartialName('System.Xml')
import System.Xml
from System.Xml import *
from System.Net import WebRequest
from System.IO import StreamReader

def getDataAsListOfXmlNodes(URL):
  request = WebRequest.Create(URL)
  request.Method = "GET"
  request.UseDefaultCredentials = True
  response = request.GetResponse()
  result = StreamReader(response.GetResponseStream()).ReadToEnd()
  doc = XmlDocument()
  doc.LoadXml(result)
  nsmgr = XmlNamespaceManager(doc.NameTable)
  nsmgr.AddNamespace( 'z', '#RowsetSchema')
  nodes = doc.SelectNodes ("//z:row", nsmgr )
  return nodes

def convertNodesToDicts(nodes):
  listOfDicts = []
  for node in nodes:
    attrs = node.Attributes
    dict = {}
    for a in attrs:
      dict[a.Name] = a.Value
    listOfDicts.append(dict)
  return listOfDicts

nodes = getDataAsListOfXmlNodes('http://host/sites/mysite/_vti_bin/...')
dicts = convertNodesToDicts(nodes)

Uploading my merged file to a document library on the server wasn’t so straightforward. I knew that SharePoint provides a set of web services APIs, so I started by acquiring the IronPython “Dynamic Web Services Helpers” from the Web Services sample. Among other things, these wrappers make it trivial to consume a WSDL-based web service. Here, for example, is a snippet that uploads a photo using the Imaging web service:

import System, clr
clr.AddReference("DynamicWebServiceHelpers.dll")
from DynamicWebServiceHelpers import *

filename = 'jon.jpg'
ws = WebService.Load('http://HOST/_vti_bin/Imaging.asmx')
ws.UseDefaultCredentials = True
bytes = open(filename,'rb').read()
bytes = map (ord, list(bytes))
bytes = System.Array.CreateArray(System.Byte,bytes)
ws.Upload('Photos','',bytes,filename,True)

So far, so good. But when I then looked for a generic service to upload any file to any document library, I found myself on a slippery slope. In the midst of exploring how to use the Swiss-Army-knife Lists service to accomplish a simple file upload, I realized I was working way too hard. Back in 2004, Bill Simser reached the same conclusion:

There seemed to be a lot of argument about using Web Services, lists, and all that just to upload a document. It can’t be that hard.

And it isn’t. As others have discovered too, SharePoint responds to a plain old HTTP PUT. Here’s an IronPython update to Bill’s recipe:

def upload(HOST,fname,rdir,rfile):
  wc = WebClient()
  wc.UseDefaultCredentials = True
  bytes = open(fname,'rb').read()
  bytes = map(ord,list(bytes))
  bytes = System.Array.CreateArray(System.Byte,bytes)
  url = '%s/%s/%s' % (HOST, rdir, rfile)
  wc.UploadData(url,'PUT',bytes)

And presto. A local file called, say, myfile.html, lands someplace like http://host/sites/mysite/MyLibrary/myfile.html.

Sheesh. If it feels like you’re working too hard, maybe you are. Step back, take a deep breath, and look for a lazier solution.

6 Comments

  1. I couldn’t agree more sometimes big hard problems have been solved long long ago. You just don’t always know or worse yet, REALIZE they were solved. I would bet also if you need it, WebDAV could be harnessed if you needed any extra functionality in the file transfer into SharePoint.

  2. I’m not able to use IronPython in a particular environment, but I still need to PUT to Sharepoint. Do you have any clues about what is going on behind the scenes with authorization? I’m especially interested in getting this to work with curl.

    I’m able to use curl in this fashion with Plone (when simply specifying a username:passwd pair), and it seems like Sharepoint should also allow it.

    thanks for any insight!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s