Unifying HTTP success and failure in .NET

In an earlier installment of the azure+elmcity series I griped about some inconsistencies in how the .NET Framework deals with HTTP:

The .NET equivalent to Python’s httplib, for example, is the HttpWebRequest/HttpWebResponse pair. But these APIs differ from those provided by httplib in a couple of ways that annoy me.

First, there’s an inconsistency in the way headers are handled. You get and set most headers using the Headers collection. But you get and set a few special ones, like Content-Type and Content-Length, using special named properties.

Second, status codes are handled inconsistently. Most responses return status codes. But for codes in the 4xx series, an exception is thrown.

To me these behaviors are quirks that make it trickier to use RESTful interfaces.

The exceptions, in particular, make it much harder to write tests. When I test the method that puts a blog into the Azure blob store, for example, I expect success, and here’s how I express that expectation:


Assert.AreEqual(HttpStatusCode.Created, response.normal_status);

But when I test the method that creates a public container, I expect failure if the container already exists. Here’s how I express that expectation:


Assert.AreEqual(WebExceptionStatus.ProtocolError, response.exception_status);

In order to deal with successes and failures in uniform way, I created an http_response_struct that encapsulates both, and a method that performs a web request and returns a structure of that type.

The code, in its current form, appears below. I present it here for two reasons. First, because it may be of value to others. But second, because others have surely done this in better and more general ways. I’m hoping this entry will attract pointers to some other simple but effective implementations of this idea.


public struct http_response_struct
  {
  public HttpStatusCode normal_status;
  public WebExceptionStatus exception_status;
  public string message;
  public byte[] data;
  public string data_as_string;
  public Dictionary<string, string> headers;

  public http_response_struct(HttpStatusCode normal_status, 
      WebExceptionStatus exception_status, string message, byte[] data, 
      string data_as_string, Dictionary<string, string> headers)
    {
    this.normal_status = normal_status;
    this.exception_status = exception_status;
    this.message = message;
    this.data = data;
    this.data_as_string = data_as_string;
    this.headers = headers;
    }
  }


public static http_response_struct DoHttpWebRequest(HttpWebRequest request, 
    byte[] data)
  {
  request.AllowAutoRedirect = true;
  HttpStatusCode normal_status;
  WebExceptionStatus exception_status;
  string message = "";
  request.ContentLength = 0;
  Dictionary<string, string> headers = new Dictionary<string, string>();

  if (data != null && data.Length > 0)
    {
    request.ContentLength = data.Length;
    var bw = new BinaryWriter(request.GetRequestStream());
    bw.Write(data);
    bw.Flush();
    bw.Close();
    }

  byte[] return_data = new byte[0];
  string return_data_as_string = "";

  try
    {
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    normal_status = response.StatusCode;
    exception_status = new WebExceptionStatus();
    message = response.StatusDescription;
    foreach (string key in response.Headers.Keys)
      headers[key] = response.Headers[key];
    get_response_data(request, ref return_data, ref return_data_as_string, 
      response);
    response.Close();
    }
  catch (WebException e)
    {
    exception_status = e.Status; 
    normal_status = new HttpStatusCode();
    message = string.Format("{0} {1}", exception_status.ToString(), 
      e.Message);
    get_response_data(request, ref return_data, ref return_data_as_string, 
      (HttpWebResponse) e.Response);
    string logmsg = string.Format("DoHttpRequest ({0}): {1}", request.RequestUri, 
      message);
    write_log_message(logmsg);
    }

  return new http_response_struct(normal_status, exception_status, 
    message, return_data, return_data_as_string, headers);
  }

6 Comments

  1. Hi,

    to mitigate the status problem we use something like (sorry if the markup’s broken):

    static HttpWebResponse GetHttpWebResponse (HttpWebRequest r) {
    HttpWebResponse result;
    try {
    result = r.GetResponse() as HttpWebResponse;
    } catch (WebException e) {
    if (e.Response is HttpWebResponse) {
    result = e.Response as HttpWebResponse;
    } else {
    result = null;
    }
    }
    return result;
    }

    (you still have to test for null, though)

    Regards,
    tamberg

  2. I’m sorry for being blunt, but that’s some really nasty C# code you’ve got there. Both in naming convention and in what it actually does. tamberg nailed it. Just catch WebException and return its Response property. With .NET 3.5 you can even build this as a sexy extension method on the HttpWebRequest class like this:

    public static class HttpWebRequestExtensions {
    public static HttpWebResponse GetResponse(this HttpWebRequest request) {
    HttpWebResponse result;

    try {
    result = request.GetResponse() as HttpWebResponse;
    } catch (WebException ex) {
    result = ex.Response as HttpWebResponse;
    }

    return result;
    }
    }

    to invoke the extension method, just do the following:

    HttpWebResponse response = request.GetResponse();

    I completely agree that it’s moronic of the HttpWebRequest class to throw exceptions when the status code is above or equal to 400. At least, it should be configurable. As an example, “410 Gone” is, after all, not an exception, but an expected response that can and should be handled gracefully.

    A simple switch(response.StatusCode) is the best way to handle all possible status codes and It’s really painful that Microsoft won’t allow this out of the box.

    PS: Comment preview would have been awesome.

  3. > I’m sorry for being blunt

    No worries, I am very green in C# and don’t pretend otherwise.

    > in naming convention

    What do you prefer and why?

    > and in what it actually does

    FWIW, this wrapper isn’t just a way to guarantee a response, but also to package up everything for downstream inspection — not just in a debugger but, frequently, from an IronPython client.

    > you can even build this as a sexy
    > extension method

    Very nice. Thanks!

  4. Pingback: Ako Help Desk
  5. Pingback: New York Snow

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s