Craig Rowe

Techlead / Developer

30th November 2009

Overengineering and a lack of testing, the pitfalls of personal projects

On this weeks show 'Content Negotiation' and browser testing

Ok, so yesterday I jumped the gun somewhat with the rather over ambitious: MVC version of up now - pretty much the same (hopefully) fleshed out notes and added link to thebarn page

Inevitably this was followed by:

@cargowire heads up for you this is what I see in Safari 4 for OK in FF3.5

Screenshot of Safari showing plain text

Clearly I'd made three (or at least three of my) classic 'personal project for self only' mistakes:

  1. Overengineering
  2. Lack of testing
  3. Eagerness to get it live

I had set out to merely replace the backend of my previous .NET WebForms project (that used XSL/XML for view separation) with an implementation of A fairly trival task, that along the way I expanded by virtue of being my own client and wanting to try out random things.

Being effectively an engine swap (I was even using the same XSL as well as javascript/css) I neglected to worry too much about browser rendering. After all, it was the same CSS that I'd tested before... surely all would be well.

The problem: Content Negotiation

My original cargowire implementation had the ability to serve different content types baked in. You could call most pages in the following ways (you still can now, try it):

  • (rendered html)

I wanted to keep this functionality with the new asp.NET MVC version, however somewhere along the way I decided it would be a good idea to not only expose these different formats through varying file extensions but also respond to the HTTP Accept header that get sent with the request. This behaviour, similar to the respond_to method from Ruby On Rails, would allow multiple versions of the content to be requested easily.

The implementation was simple and based on my brief reading of the Http Accept header description.

            // Pseudocode of the algorithm
            format = request.extension
            if exists(format) and haveViewFor(format)
              return content as format
              read in list of values from Accept header
              sort the list of accept headers by quality first and original request order second
              return content as the first format in the list that matches an available view
            if no appropriate view found
              return content as html

By attempting to find any possible view it allowed me to later add any format I like (json for example) just by adding the relevant view and all the plumbing would wire up.

All seemed well in Firefox, and after forgetting about this code and finishing the rest of the backend off, browser testing was irrelevant.. right?... I'd not changed any css, html or JS... I pushed live.

Why did Safari looks so funny?

During testing I'd been working with firefox, which, if you request an extensionless url, appears to send the following accept header (according to the live headers plugin):

            Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

So in my algorithm you'd get:

  1. text/html (default q = 1) - sorted before xhtml because requested first
  2. application/xhtml+xml (default q = 1)
  3. application/xml;q=0.9
  4. */*;q=0.8

In this scenario the code looks for a template for html, finds one and returns (ignoring the rest of the list).

When I saw the output from porkandpaws tweet it was immediately clear that he was seeing the XML output instead of html. A quick run in debug mode gave the following accept header from Safari 4:

            Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

It would appear that if no extension is specified Safari will prioritise XML over XHTML. With application/xml requested before application/xhtml+xml and with both being assigned the default quality level 1 my algorithm assumed the client wanted the XML view and returned it!

The quick fix was to prioritise types containing 'html' before falling back to request order. This may not be a perfect solution, but lets be pragmatic, I'd only implemented this on a whim - no-one is actually using this site other than for the html or rss view.


To round up then, although overengineering and random play code is almost inevitable for the curious developer when working for themselves with no timescale, it doesn't excuse full testing and you can never rule out what any change, no matter how 'isolated', might effect.

Other Recent Posts

Workshop: Hack Yourself First(18th July 2017)


re:develop 2016(14th October 2016)

Having not made the trip down to Bournemouth for re:develop 1 or 2 it was great to be in attendance for it's third incarnation.…

DevSouthCoast GameJam 2013(15th September 2013)

Making games over a weekend... competitively... and we chose a dead technology... why the hell not!…

NodeCopter Southampton 2013(11th August 2013)

Hack Days are awesome. How could they not be? you get to make stuff with like minded people with no bosses, no client deadlines, no point but the love of it. …

Joining Dootrix(24th November 2011)

It's been a while since I posted. I'd like to say that's because a lot's been going on. In reality I got lazy and now I just happen to have something to write about that can make it sound like a lot has been going on.…