I understand that to measure a project or code, we can use The Joel Test, but is there any simple standard test (like The Joel Test) that is able to measure and filter how good a programmer is?
My plan is to have this test as a quick filter first before going to a more detail test.
There is the programmer competency matrix.
As with the Joel test it's just a vague guide. The only way to properly assess a programmer is to ask good programmers who have worked with them.
I would turn the Joel test around:
Have they used source control?
Do they know how to automate a one step build?
...
The only one that doesn't seem particularly applicable is the have testers question. The others that seem kind of off turn in to a this is how we handle it how have you handled it in the past sort of questions(This is how we handle keeping our schedule up to date how have you handled scheduling in the past?).
edit:
Basically you don't get the stuff in the Joel test for free, you have to hire people who can make it happen. You want to establish their ability to make that happen.
The Joel Test is just an informal baseline check to quickly judge whether a place has good working conditions for programmers. Even if it scores a perfect 10, it can still be a hell-hole that is going to go bankrupt six months down the road. A low score is an indication of something not quite right, and makes for excellent interview questions ("You're not currently using source control; are there any plans to do so in the future?"), and the answers might be such that you'd accept the job despite a low Joel score.
The Joel Test is not a 'standard' test either; it's just a checklist Joel Spolsky posted on his blog.
As far as 'measuring' the quality of a programmer goes; unfortunately, the really important skills and qualities of a good programmer are hard or impossible to quantify, so there is no replacement for thorough human evaluation. You can weed out the completely clueless candidates quite easily though, using a very simple programming task - ideally, something that involves recursion, tree structures or pointers (a programmer who doesn't 'get' these is unlikely to be of much use). For those who pass this test, you'll have to evaluate skills manually: read code they wrote, test drive applications they wrote, give them more programming tasks (both design and implementation), watch them work, talk to them, see if you can spark a professional discussion. If you're looking for a specialist / language guru, you might also want to ask a few knowledge questions, but for a general programming candidate I wouldn't bother.
Yeah:
Do you program in your spare time?
In all my experience, this single question is most indicative of how good a programmer is. If they enjoy it; if they have passion for doing the task, then they're going to be good at it.
And frankly, a lot of 9 to 5 jobs don't involve a lot of coding. They don't involve a lot of iterating through the life-cycle of designing new programs and seeing how that design works/fails. Without that iteration, there's simply not the practice needed for programmers to gain core program design skills.
And they don't involve a lot of learning. Programmers who even simply hack on things at home are going to be exploring new and interesting solutions without the constraints of big business.
It's not as detailed as the Joel Test, but asking them to write a fizz buzz program will be a good indication to see if they can code at all.
http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html and http://imranontech.com/2007/01/24/using-fizzbuzz-to-find-developers-who-grok-coding/
That will not tell you about the software engineering maturity of the individual, but it will screen out the worse.
Eh, I have qualm with the wording at the start. It's not "have you used X" or "Do you know of Y", it's a matter of actually using and doing. Any programmer that hasn't touched or heard of the items on the Joel test is simply disconnected and needs to get a clue. But you're right, code-shops fail the Joel test because people in the shops let it fail. The only defense I can see runs along the lines of "I tried, but did not have the authority. And now I'm applying here".
Do you use source control?
Yes, but
Can you make a build in one step?
Do you make daily builds?
Do you have a bug database?
Yes, but I haven't configured it and I don't administer it, I simply use it.
Do you fix bugs before writing new code?
Do you have an up-to-date schedule?
Nope, that's not my job.
Do you have a spec?
I get given a spec, I then analyse it and produce relevant documents.
Do programmers have quiet working conditions?
Do you use the best tools money can buy?
You don't know what the best tools are and if you think that you do, there will always be someone to argue your point.
Do you have testers?
Yes. Actually, yes and they are not very good, but that wasn't in the question.
Do new candidates write code during their interview?
Yes and they fail. Yes and they pass. What does this tell you?
Do you do hallway usability testing?
No, but what if we do something better?
To conclude:
This is not a rant, but I would be very interested to hear what kind of developer you think I am based on the answers that I have provided. This will hopefully prove my point.
