21st February 2010
The application process is there for two reasons. One, for the employer to get to know if the employee is a good fit. And second for the employee to get to know if the employer is a good fit.
With any role a new hire needs to address a number of 'transferable skills' questions such as 'can this person communicate clearly' and 'can this person work with others'. However, on top of this each role has specific skillset requirements. It is these requirements that need bespoke methods of identification. The question is, what methods should we use?
Developer application processes can vary greatly. However their focus must be 'is this person able to program at the level we require?'.
Here's just a few examples of the methods I have encountered myself:
The Joel test (a means of measuring the quality of a software team) provides a pretty compelling argument as to why candidates should write code as part of the application process.
Joel Spolsky - http://www.joelonsoftware.com/
Do new candidates write code during their interview?
Would you hire a magician without asking them to show you some magic tricks? Of course not...
...Yet, every day, programmers are hired on the basis of an impressive resumé or because the interviewer enjoyed chatting with them. Or they are asked trivia questions ("what's the difference between CreateDialog() and DialogBox()?") which could be answered by looking at the documentation. You don't care if they have memorized thousands of trivia about programming, you care if they are able to produce code. Or, even worse, they are asked "AHA!" questions: the kind of questions that seem easy when you know the answer, but if you don't know the answer, they are impossible.
Please, just stop doing this. Do whatever you want during interviews, but make the candidate write some code.
So how do each of the approaches above answer the big question "Can this person actually develop software with us / the way we do / the way we intend to in the foreseeable future."
If we take a look back at some of the variants only a few actually ask for code:
|Can you develop?|
|Mini Contractor||At the end of the day the employer sees your code and assesses how much help you needed from others||In this case the company needs some work for you to do that you can get up to speed with quickly (rather than wasting time teaching you) and is not under NDA. This approach can be useful to see how the applicant works in your environment but has extreme time costs for both parties, particularly as their presence may disrupt 'non-interviewing' members of staff.|
|Blue Chip (legendary logic tests)||Answer a thought experiment/riddle||I've not encountered this myself during interview - the concept is almost mythical. Although logic questions may, like apptitude tests, give an insight into how 'smart' someone is it is nowhere near as directly related to programming as getting actual code samples to be done/provided.|
|Right here right now||Produce a code sample to a particular problem in a short time frame while being monitored||Requires a relatively 'trivial' task that can be completed in a short amount of time. Being watched adds abnormal pressure but maintains control over the entire process and removes the ability to cheat the system. If the situation doesn't involve a computer (code on white board) it is quite different to the expected role.|
|Show us the code||Produce a code sample to a specific set task prior to coming in for interview||Can be 'gamed' (completed by someone else and/or with abnormal effort - 'yeh it only took 2 hours *cough*every minute since you gave me the task to now*cough*'). Doesn't give you a direct insight into the process only the process as viewed by the participant retrospectively.|
If we agree that completing real code in some way is the only way to identify the programming ability of a prospective candidate we rule out all but the last two approaches (with 'right here right now' and the 'mini contractor' being roughly equivalent but for the probable time frame involved). And then it becomes a case of which is better?
At Headscape we use a process similar to 'Show us the code'. After a CV/Covering letter and phone interview sift we ask applicants to complete a mini project (based on a spec provided) and submit it to us as the next stage. This is for a number of reasons:
The argument is essentially the same as the coursework vs exams debate from education. Although coursework is more open to cheating and less able to be tightly controlled it is actually more like working life.
Discussing the project can easily identify if the candidate has had some 'assistance' from a third party. The main shortcoming then is not knowing exactly how much of the time between phone interview and in-person interview the candidate spent creating the output they provided. In my view this shortcoming is balanced by being able to have an in-depth discussion of the project. The speed of response and passion in their discussion surrounding this should provide pointers to how 'smart' they are.
Any suspicions or a large amount of applicants can be filtered by doing an initial technical interview over the phone regarding the project if necessary.
So thats all well and good as an employer. Just sit back and fire out homework for your applicants. But as an applicant is it merely another hoop to jump through?
When I've been an applicant myself I actually prefer to do code in some way. Not because I like to be tested or even just because I like to write code but because it allows me scope to get a feel for the employer.
All of these points allow me to guage my half of the application bargain "Is the employer a good fit for me?".
While agreeing with Joel that Smart does not mean 'knows the answer to trivia questions' I also understand that by asking for a mini-project to be completed we are asking for a reasonable amount of commitment from the applicant. However we both benefit from this in the richness of the interview process that follows.
Something we have also considered at Headscape is changing the 'test' to a situation where we provide a solution to the applicant and ask them to extend it in some way. This would be more realistic (most projects are continuations of old ones, or built on a base codebase rather than clean slates) and would also allow us to ask the applicant their thoughts on the project we provided. Would they have done it this way?
Allowing them to critique us not only assists their desire to judge if we are a good fit for them but allows us to see their analytical and critical skills.
Our interviews involve key personal (representatives from each team PMs, Directors, Developers) and we must assess the standard job interview issues. However when it comes to development some form of coding is essential. In our case we favour the 'coursework' style. It may be that a mix of both is appropriate. With all things 'it depends'. If you expect 100 applicants to get to the testing phase then assessing those projects will be a huge undertaking. But relying on only a small in-person algorithm test can be difficult to generalise potential success from. Mixing the two, by bringing in a candidate and asking them to code all day may seem like an ideal compromise but requires a single fixed large timeframe commitment from both parties. We also all know how different it is to code with someone watching over your shoulder.
At all times we need to consider the benefit for both us as employers and applicants as potential employees through the process. We are not the only ones making a big decision. If we can somehow forge the process to allow the applicant to test us we will inevitably have a better new hire in the long run.
All article content is licenced under a Creative Commons Attribution-Noncommercial Licence.