29th August 2010

Thoughts on Common Code...

What?
Why?
Thinking Further
What more can you do with it?
How to do it?
Downsides?
Conclusion

As a developer in the web industry you're probably used to buzzwords that seem to get clung on to and overused by non-techs. For me, one of the most oft heard is 'Generic'. An all encompassing word that encapsulates the hopes and dreams of non-techs that as soon as some work is done once, it will never have to be done again and can easily be part migrated into any and all projects, old or new.

Of course, this isn't the reality in a lot of cases, but it should be the gold standard. There's little benefit in constantly reinventing the wheel or working in isolation from the world around you.

What?

So let's start from the beginning... If you're a developer who went the traditional education route you'll be well aware of the idea of reusability.

Reusable code is code that can be used, without modification, to perform a specific service regardless of what application uses the code. http://msdn.microsoft.com/en-us/library/aa189112%28office.10%29.aspx

By using a CMS we are using someone elses 'reusable code', by building on top of a framework, or even using someone's scripting language we are all running at some level of abstraction through reusable code. We are at the application level - few of us are creating languages but many of us are creating bespoke applications.

The languages and frameworks we use don't often provide application level functionality, instead they provide the bones with which we make the skeleton, and often they provide us with multiple options.

So we are at the next level, the applications themselves. Does this mean we don't create reusable code? that we sit atop the mount of others? Certainly not.. As a web agency we work with a variety of clients in a variety of industries but if we think about it these aren't too different.. We've got CRUD operations, entity/list manipulation, email, security, performance considerations.. all of which could be said to apply to almost any project and so should be kept separate.

Why?

So let's think about why we should be maintaining our own common codebase on top of all the other peoples code we are reusing... starting by breaking down a view of project code:

Fig 1.0: The world of project code.

The above diagram indicates the categorisation of code I like to consider when developing a website/application. A codebase can be broken down relatively simply into:

Level 1: Code that plugs holes in the language or framework
Level 2: Code that could in theory be used by any website
Level 3: Code that is specific to a particular client, for example, they may have a bespoke system (human or technological) that you are integrating with that no other client will
Level 4: Code that is specific to the particular project or problem that you are working on

You could go even lower, although I won't for the purposes of this post, and say that beneath level 1 there are the design patterns and algorithms that you favour that reach further than even one language choice. Equally you could suggest that in parallel to this world is the breakdown of code in competing company codebases. But before someone mentions Schrodinger's cat lets get onto the benefits of keeping a common codebase. Starting with the most obvious and hyped...

Reuse

One of the most important things to note is that No project is bespoke. No project sits solely on that outer rim of the world even if it sits atop an off the shelf system at the lowest two levels. Often there is very little about a project, from a development point of view, that is actually bespoke. If your site is purely content you're talking about a CMS. If you are integrating with a third party app there's a degree of wrapper code that is distinct from the project specific application.

To take it further if you are manipulating data in some way, outputting reports in CSV, encrypting some data or interacting with a database this should be created free from the rest of the system and reusable.. to take it to the extreme, outside of UI wiring, some projects could be made up almost solely of calls into a common codebase.

Speed

If you are reusing code to that degree your development times will be that much quicker. Need to two way encrypt something? nip into your common encryption code and your done.

But let's think about this some more:

Thinking Further

Quality

If you are constantly reusing code you will also be constantly fixing it, refining it, testing it in different contexts and scenarios. You won't finish a project and not look at its codebase again until a year later when some obscure bug gets reported.

This has obvious ties to maintenance and upgrades. If you find a better or more secure way to do something it may be as easy as updating the common libary version used by projects. This is less possible if developers have rolled their own solution to all problems. 'Oh our encryption isn't very strong. I've updated my method, did you use similar?'... 'yes but I called it something else and it's mixed in with the integration code I did'... 'oh'

Knowledge sharing

If you're working within a team there's always the concern that you might be redoing the work or research that someone has already done. Short of asking around everytime you embark on the smallest of algorithms you can search the common codebase (perhaps paired with a team wiki) allowing others to get on with their work while you benefit from the shared knowledge base.

By this means you are capturing and sharing the knowledge of prior team members, contractors and others who are not in the office that day.

Process sharing

Short of coming up with a programming style guide (that you toil over for weeks until realising no-one will bother to read it) the common code library acts as a vehicle for convergence among the different programming styles of the team. As you each add and modify items within the common code you come to an understanding as to naming conventions, architecture and style purely by absorbing the pre-existing approach and fitting additions around it. What's more this isn't seen as a convention that is pushed upon the team nor is it autocratically defined by a single lead. It grows and evolves.

Expectation setting

If a new team member joins they could trawl through old projects to get a feel for approach or they could sit in a meeting or training day, or even be assigned a mentor. Ultimately though, providing them with a good chunk of reusable code is an easy expectation setter. It allows them to quickly see and guage the combined wisdom of the team, as well as acting as a base that they can expect to see in projects they are assigned to work on. Indeed, if they are particularly junior you can consider the common code base an effort in avoiding bugs as you are providing them with a big chunk of, in theory, bug-free code.

Furthermore it can provide a natural dividing line where code reviews can take place (at the gateway into the main common code branch). It can also be seen as a performance indicator. At the end of a project you could rightly expect to either find a lot of calls into the common code or a lot of new common code. If not, why not? Is the project really that bespoke? were there contributing factors (lack of design time, a short terminist perception of 'not enough time') that need to be addressed? Even better, have they found issues with the common code and applied fixes or updates?

Design enforcement

Ultimately by having a common codebase and by categorising the code you create into the four levels you are forced to ask questions of yourself as you code. 'Can i extract some of this?', 'Would this be beneficial to others?', 'How can I better abstract this?', 'Is this likely to be needed by all the clients apps or just this one?', 'Is this of good enough quality to sit aside the rest our common code?' and perhaps most importantly 'If it's not in our common codebase is it in someone elses?' Why am I not looking for and using that? (and if it's not out there you've just found a gap to fill and a question you can answer on stackoverflow).

What more can you do with it?

Outside of your own team/company if you've built a set of reusable components/classes/functions, and you've done this in a way that you can easily segment into the various levels, you could decide to put it out there, open source it as an addition to the public space for your stack of choice (with the bonus that at the first two levels you can be sure it's not even putting client confidentiality at risk).

How to do it?

Set aside a central area in source control (you're all using source control right?)
If you're creating it for the first time go back over some projects and extract as much as you can
- Even if you don't refactor that project to use the common code just yet at least you've pulled the knowledge out of it for future use
Ensure the team are completely aware of its location and how it's to be used
Ensure there is a review process in place
- Perform code review for additions and rolling reviews of the code in relation to changing standards/upcoming tech
Decide upon an organisation method
- Within the .NET world I prefer to mirror the System namespace with my own company name. In this way any consumer already has a frame of reference for where they would expect to find something.
Keep it up to date
- This may even include eventually delegating calls directly to the framework if they plug the holes you plugged some time ago

Downsides?

Of course as with anything there can be perceived downsides. Maybe there are battles against management, against short sightedness that you have to overcome. But this is where the senior devs have to take on the internal salesman role.

Pragmatism, though, must not be forgotten. Don't go around suddenly making every minor thing a common code class 'just in case you want to swap it out later'. Be realistic, keep in mind good programming practices... SOLID, DRY, KISS and YAGNI etc.

Conclusion

People talk of Utility Belts, and Reusable Code and fret over it a lot. Ultimately any good developer should already be thinking about reusability and more generally the idea of a common codebase. The key is to decide upon a good structure, a method of introducing it to the team and a plan for keeping it ticking over. As well as being aware of the less hyped benefits such as performance indicators, training tools, design enforcement and encouragement to reuse external code (and provide yours for reuse by others).

Embrace your common codebase. It's more than just some utility functions in an include file.