From my experience in my company where I did many interviews, there is a good chance that the person interviewing has no clue on how to do that properly. So they prepared a set of technical questions and calculate a score of that and make that your resume. This however has many drawbacks and should not be done for the following reasons:
You ask point knowledge. If the programmer happens to never have done something in that area, he/she still might be an excellent co-worker, but just doesn't know that particular answer. On the contrast: if someone had prepared for the interview and found the answer to that particular question on the net, you get the right answer, but that person might not have a clue about the actual topic at all.
People are nervous in job interviews. The brain has this great feature to shut down many higher-level areas (like logic) if in panic, which means: if you are nervous, you might not deliver the quality of answers you would in a day-to-day situation. Some people can deal with a stressful situation like an interview, many can not.
With a single, correct answer, you test that persons skill to find that particular answer. This is one of many skills an it co-worker needs, but not the one and only required. Therefore one or two of those questions should be enough to test that area of knowledge, and then other skills should be queried. An interview that only contains problem-solving questions tests the same skill over and over again.
What are good programming tasks questions?
Those famous "Can you write a short program" questions have the huge problem that most programmers cannot write a single line of code without their IDE helping them. But that is in day-to-day working situations no issue at all, because the programmer always has his IDE helping him. So asking things like "Find the error", "Write 50 lines of code that do..." or even simple questions need to take into account, that the applicant does not have his tools (IDE, Google) available.
I for example can answer you basically any question within 1 minute if I have Google helping me, but without Internet connection I appear to be helpless. I call that outsourced memory, and instead of hindering me, it very much helps me to focus on what is really important - understanding the underlaying mechanics - because everything else can be looked up. But don't ask me about details from any random APIs, because I don't know those, I have Google for that.
That said, a good programming task question should not focus on knowing APIs, or special coding skills unless this is an absolute requirement for the job. Knowledge can be gained, so it is better to find out how good that person is in gaining knowledge than to ask what he/she already knows.
A good question for a programming task should be short, simple, able to be coded in every language with only a few lines of code and it - especially - should tell you as much as possible about how the person works and finds answers. Example:
"Write a function in the language of your choice that takes an array of integers and re-orders them in a way that the first integer is afterwards the last, and all others shift accordingly."
The first any applicant should ask at this point is: "Sorry... can you please explain the task?". Because no programmer has been given a clear description on what to do ever. This is followed by the explanation, that the code in questions should do a left-shift of the content of the array with the overflow being added to the right.
This task is so simple, that anyone who graduated any form of programming level should be able to answer properly. This takes into account that the programmer has to work without his tools and that being nervous reduces the ability to think logically. It however still tells you how people solve problems just from the way the question is worded and from the way how people approach it, simply because a left-shift is against the common 'left-to-right' instinct and forces people to think for a second.
There are many possible answers to this question, so taking a close look to the way the code is developed is the important part, not if the solution would actually work. Does the applicant test for null? How is the overflow stored? Is a loop or a mem-set used? How does the applicant verify code correctness? This one simple question tells you an entire biography about how that person works.
What are good general knowledge questions?
Good questions are easy to answer, allow for a broad amount of answers (so called 'open questions') and allow you to learn as much as possible about the applicant as you can in the short time you have.
Examples:
(Asking a C++ programmer): "What other languages beside C++ do you know?"
This is an entry level questions, which gives the applicant a fair chance to bail out at this moment should he not know anything about the topic asked. A 'no' at this point is better than to torment him/her with several more questions that he/she all has to answer with: "Sorry, I don't know anything about that."
I addition it tells you first of all which other languages that person knowns, furthermore you learn how interested that person is to get a more broad view of the programming world, or if you have someone with only a singular language (and therefore feature/techniques) view.
(Next after that, lets say he knows Java): What are the top three differences between C++ and Java in your opinon?"
This is an open question that allows for many answers, so the applicant has a good chance to find at least three. Asking for the (personal opinion) top three not only limits the possible answers, but also forces the applicant to sort based on priority. Still it is (or should be) easy to answer.
This is a simple question that tests a lot of in-depth knowledge about different programming languages. How deep is the knowledge of those topics really? From those answers you can tell a lot about the knowledge and actual understanding of the underlaying mechanics of programming languages. How much that person spent with the dirty details, or if he/she is just someone who links together various API functions with no real clue what happens beneath those.
This concept of entry-level questions followed by simple in-depth knowledge questions can be used for most other topics as well. Always in this scheme: bail-out question, verification question, in-depth question. Another example (from a Java interview):
- "How would you rate your experience with multi-threaded development?"
- "Please name what you think are the top three most important things to consider when developing a multi-threaded application."
- "Please name three classes from the Java API that can help you on developing those applications and give a short description on what they are used for."
These three questions will tell you more than any technical question what the applicant really knows about those topics, while being fair to answer considering point-knowledge and stress level.
So the next time someone asks you 20 coding questions in a row, you know that he or she has basically no idea on how to interview someone properly. ;)