15

I've recently been in an argument with a colleague about technical questions in interviews. As a graduate, I went round lots of companies and noticed they used the same questions. An example is "Can you write a function that determines if a number is prime or not?", 4 years later, I find that particular question is quite common even for a junior developer. I might not be looking at this the correct way, but shouldn't software houses be intelligent enough to think up their own interview questions? I've been to about 16 interviews as a graduate and the same questions came up in about 75% of them. This leads me to believe that many companies are lazy and simply Google: 'Template questions for interviewing software developers' and I feel there are doing themselves a disservice in taking this approach.

Question: Is it better to use a set of questions off some template or should software houses strive to be more original and come up with their own interview material?

From my point of view, if I failed an interview and went off and looked for good answers to the questions I messed up on, I could fly through the next interview if the questions are the same.

Desolate Planet
  • 6,038
  • 3
  • 29
  • 38
  • 1
    Out of curiosity, where did FizzBuzz style questions originate? – Desolate Planet Sep 01 '11 at 18:50
  • 1
    The software house has to be careful not to get too involved with the question that they become the person that has spent thousands of hours perfecting the solution that most people would take a few days to get unless they have been with the company and seen the question already. – JB King Sep 01 '11 at 20:28
  • 2
    Use template questions, if you're ok with template answers. – back2dos Sep 28 '11 at 15:59

8 Answers8

19

A lot of those sorts of questions stick around precisely because it is hard to come up with questions that are clear, require as little domain knowledge as possible, require an ability to come up with an algorithm without being a trick question that someone will either "get" or "not get," and that people have a historical background of answers to compare the candidate against. And the best questions have multiple levels of correct answers that allow the interviewer to gauge the candidate's problem solving skills. If someone suggests a trivial prime-finding algorithm, for example, we can then talk about how you might optimize the implementation or the interviewer to lead the candidate toward one of the more sophisticated algorithms.

I've been in groups where a number of different developers rated how difficult a particular problem would be for a set of interviewees and the estimates were all over the map. Lots of people look at FizzBuzz initially and dismiss it as patently too easy. If you go into the interview without a good understanding of how difficult the questions are, the fifth person you interview has a massive advantage over the first person because you've learned that people struggle with FizzBuzz while you formed a negative impression of the first person you interviewed because they had identical struggles.

If developers are coming up with fresh questions, they also tend to have unintended cultural or domain knowledge embedded in them. They'll ask questions about a sport, for example, that tends to give an advantage to people that know the rules of that sport who can either quickly spot an ambiguity or make an "obvious" assumption. Someone that is utterly unfamiliar with the sport has to figure out the rules of the game on the fly and is much more likely to make a mistake or to come up with a less sophisticated answer. Or they'll ask a question that unintentionally assumes that candidates will know/ remember some relatively minor tidbit. Someone that has been working on accounting systems for years, for example, may assume that everyone knows what a double-entry accounting system is or that a question that assumed a simple double-entry accounting system schema definition would be clear to a competent developer. But it could easily throw a good developer that has to get their head around the idea of a debit to cash increasing the cash balance and a credit decreasing the cash balance. Someone that has taken Accounting 101, however, would quickly catch on. If your intention was to test the candidate's familiarity with accounting, this might be a perfectly reasonable question. But if you're not looking for a subject matter expert, it would be problematic.

While I suppose it's possible that someone would could memorize the answers to the few hundred "common" template questions that people use, that doesn't bother me much as an interviewer. In the first place, it would be somewhat impressive if they did so and would reflect quite a bit of industriousness and problem-solving. In the second place, it would generally be pretty good training-- if the developer knows how to reverse a string, knows some algorithms for finding primes, knows how to write loops and implement control constructs for FizzBuzz, knows the various graph algorithms that come up regularly, etc.-- they're likely a pretty well-rounded developer. If the developer managed to memorize the answers without learning anything, they'd be caught out when the interviewer wanted to drill down a bit or they'd be disqualified when the interview moved on to more open ended questions about their previous projects or delving into details about the role.

Justin Cave
  • 12,691
  • 3
  • 44
  • 53
8

I think there are those easy questions fizzbuzz, reverse a string, etc., and then there are certain questions whereas if you have not seen them before it is tough to get the intended optimal solution with the additional pressure of the interview and the short time constraints. The worst part is that sometimes recruiters with no technical knowledge ask some of these questions and they are looking for a very specific answer.... If you do not give the optimal answer they think you are an absolute idiot. Even if your answer works, solves the problem, and while it may not be optimal it is not too inefficient....

Some Examples:
How would you shuffle a deck of cards? They are looking for the Fisher-Yates method http://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle other answers are "wrong"..... This is not something that you are likely to know unless you wanted to shuffle cards before and specifically looked it up. You might stumble upon it in the course of doing the answer, but from what I have seen they expect you to know this cold so it is doubtful there will be much time given to allow finding the solution. Another less efficient (but maybe more obvious) way is to create a new array with a pointer to the original array and a number which is a random number then sort the array by the random number. Then use the sorted array to construct a new array of cards. In any case, I got 0 credit for presenting that to a recruiter.

Another example is the question about how you would detect a duplicate number in a list of n numbers all from 1 to n-1. The obvious answer (which is relatively efficient in a time perspective) is to use a hash table to insert each element and if you find one already inserted then you found the duplicate. The optimal answer is because the numbers are between 1 and n if there are no duplicates, you can get the expected sum as n(n+1)/n. Then if you sum up the array you get the actual sum which is n-d less than the expected sum. So basically n - (Expected Sum - Actual Sum) = duplicate number... It's a pretty special case.... I actually saw something before in some online post on interview problems ages ago about a related one with the same trick, so I was okay....

Another one, reverse all the words in a string without using any additional space. I had to think about it and the recruiter was pretty impatient in the 5 minutes I thought (splitting and re-joining the string in reverse, or going word by word into a new array are both very easy). I came up with finding the border of the last word, finding the end of the first word, and constantly shifting the entire list down by 1 and then inserting the appropriate letter. Then repeating until you've reached the end of the list. This method worked and the interviewer agreed (he was a bit technical), but it is pretty inefficient. When I got home I searched for the optimal answer, and it is to just reverse the list, then reverse each individual word. Some people might come up with that, but in five minutes with the pressure of an interview and never having seen the question before, I think it would be difficult....

Now it's true once you have seen some of these problems you can solve them easier. Because there are some similar questions with some of the same tricks. I know especially the n(n-1)/2 formula and number list has several variations. But still I don't know what these questions test. A FizzBuzz is something that everyone should be able to do (although i have seen variations that are not so simple in which case I start to question if even that is realistic for an interview situation without being able to type in/debug the code). Some of these questions are obvious once you see them, but if you haven't seen them they are not obvious. After all someone mentioned it took years for the first correct implementation of binary search to appear... Right now binary search is so obvious because everyone can read about it. But earlier it was a challenge to discover/implement it.

Nevertheless, I think the worst part is when non technical people are asking the questions because they cannot appreciate your solution is correct although not optimal. They just know your solution is not the one presented and hence it is all wrong, you get no credit for the attempt. Even non optimal solutions often show a knowledge of programming constructs. Unless I am programming poker games, I do not care how well someone can shuffle a deck of cards. And even if I was, after showing them the efficient algorithm I'm sure anyone halfway decent could follow it.

Template questions appear to just give an advantage to candidates who have been interviewing longer as they are more likely to have seen more of the template questions. Even FizzBuzz, the first time you see it you might freak out, but as you see it over and over again you become more used to it. The best thing you could do is give some coding challenge that requires custom business logic. E.g. create some dataset (array of objects/records) and then create some business rules to apply to each object and return an answer.. e.g. an some type of rating, etc. but very custom rules). While they have probably seen loop through records and apply logic a zillion times, your rules will be unique, so at the very least they would have to understand and successfully implement the rules you created.

Let's say projected Salary and a list of candidates and their top 3 video favorite video games, and perhaps a starting salary. Candidates who like Zelda get a 300 penalty, candidates who like the little mermaid get a 200 bonus. Candidates who like Donkey Kong and Super Mario Brothers but not Dr. Mario get a 300 bonus. Candidates who like Metroid or Kid Icarus get a 200 bonus, etc.... It seems a bit insane but it would show you their ability to translate business rules into logical program constructs and test their understand of boolean logic as well. Overall not much different than fizzbuzz except not using modulus and loops. You could of course give them the list and have them loop through it and print the results to the screen using specific output criteria as well.

Cervo
  • 1,748
  • 16
  • 16
4

On my last job search, I was in a very similar situation. I think I had the same linked list question at all companies but one (the one I'm currently employed at).

One thing I did notices was that these questions were given in the first interview.

From the interviewers point of view, if a person can't get these questions right, then they didn't prepare for the interview, so why waste more time with them? I don't think this is the right idea though. The company will get a whole bunch of people who can recite answers from Google moving on to the next round, but not necessarily a group of people who are competent developers.

I found the boilerplate FizzBuzz questions annoying personally, but it made for an easy interview. I like a challenging interview. When I interview potential employees I like to make sure I know they can think and solve problems. I expect that they can use Google.

Should a company have a question template? Yes. It helps evaluate candidates for the first few rounds. Should the template be made up of generic problems that can easily be found through Google? No. That just short changes everyone involved.

Tyanna
  • 9,528
  • 1
  • 34
  • 54
4

"Should software houses strive to be more original and come up with their own interview material?"

Yes they should, but practically what happens is that you come into work and you find an email from HR saying that you are on the interview loop for two candidates tomorrow. You are already up to your ass in alligators, and your manager will not thank you for taking half a day to come up with some new, good, interview questions. Being pressed for time, you fall back on questions you've been asked, or that you can find quickly on google.

Charles E. Grant
  • 16,612
  • 1
  • 46
  • 73
  • 2
    yes and then you're really happy you have a document full of good interview questions whereas otherwise you might be staring at the guy struggling for something meaningful to ask – Doug T. Mar 02 '11 at 00:10
3

Questions like you mention are normally fizzbuzz questions and are not a question used for hiring; they're a denying question (ie. correct answers won't get you a job in any way, but wrong answers will get you tossed in the trash). So they aren't very valuable.

It's the questions that come later that need thought put into them and will, more often than not, have some strong relation to the type of work that the company does, and what the team dynamic is like.

So the answer is: both.

Steven Evers
  • 28,200
  • 10
  • 75
  • 159
  • 4
    I agree that the fizzbuzz-style question are denying questions... but don't knock keeping the wrong people off the bus! – Peter K. Mar 01 '11 at 22:38
2

My interview template in pseudo code:

if question(FizzBuzz_type_question) == correct then

for each (question q in array_of_more_interesting_questions) question(q) else

end interview.

Jim In Texas
  • 1,453
  • 1
  • 11
  • 12
2

If most or all of their questions are the same, it is indeed a bad thing (for them, that is). That probably shows that the interviewer is not well prepared for the task.

Note that using "template" entry level questions like the FizzBuzz test may be acceptable to weed out the totally inept quickly, without investing too much time and effort. Although even then, variations on the theme are useful to avoid hearing answers straight out of Google. Then again, it is easy to ask the candidate what the code just written actually does and why it works (or doesn't). I believe that a technically proficient interviewer can quickly detect whether someone just pretends to know the answer.

And more importantly, if someone passes the entry level test, the harder questions should come, where it is not enough any more to recite canned answers. I personally am more interested in the interviewee's thought processes and problem solving abilities in general, rather than the specific solution to a riddle. The other important topic would be development processes and approaches - I initiate an open ended discussion rather than concrete questions on these topics. There are no right or wrong answer on these subjects, so no possibility to prepare in advance. But from a discussion I learn a lot on whether I and the team would be comfortable working with this candidate.

So some limited amount of "template" questions are OK, but a good interview is a flow process, where the interviewer should adapt to the situation, including the candidate's (advertised and real) skill level, mental state (nervousness / tiredness) etc.

Péter Török
  • 46,427
  • 16
  • 160
  • 185
  • Yeah, but say you give the FizzBuzz test so someone who is not suitable and they fail it. They could go look up good answers and pass that question in another company, this is the bit that bothers me and I think companies should have an obligation to not use template questions. Any good senior software developer should be able to think up good questions, ranging from easy to hard. – Desolate Planet Mar 01 '11 at 22:30
  • 2
    @Desolate Planet: Thing is though, I suspect that the people who fail fizzbuzz aren't the type to look up the answers... they'll just hope that they don't get the question asked at the next interview. – Steven Evers Mar 01 '11 at 22:31
  • Hmm..but even after they mess up 3 - 4 interviews, they will be smart enough to detect patterns in questions that are put to them. Again, this is just a thought I've had about technical interviews in general and it strikes me as a poor way to go about it. – Desolate Planet Mar 01 '11 at 22:34
  • 1
    @Desolate, then they may manage to get a job at a company which apparently doesn't even care to filter job applicants well... so both get what they deserve. – Péter Török Mar 01 '11 at 22:41
  • @Peter, I agree with what your saying, but it's rather unfair on other applicants who may be right for the job and have something they can contribute to the company. They might not get the chance because someone has looked into great answers and waltzed through the process. I'm not saying all companies are like this, but usually when I join a company, they have a word document on the network drive with a list of interview questions and they look pretty much the same. It just strikes me as very lazy. – Desolate Planet Mar 01 '11 at 23:04
  • @Desolate - But the whole point of these sorts of questions is that they're easy enough that someone that is right for the position will get them correct (or correct enough) whether or not they've studied. You're far more likely to be unfair to the right applicant when you make up your own question with no history of how previous candidates have done and no feel for where strong candidates are likely to make mistakes. – Justin Cave Mar 01 '11 at 23:39
  • @Desolate, of course it is unfair - first of all for the company itself. They won't get the best possible workers, which will greatly damage their business in the long run. But if they themselves don't see this as a problem, who are we to tell them? I am not worried so much about the applicants - good developers will be able to find a job, probably a better place than this one would have been. – Péter Török Mar 02 '11 at 08:28
1

I think the interview should be divided into:

  • Social aspect (aside from HR stuff) - Does this person fit into our organization well? Does he have the right attitude - Is he willing to follow our management style? This needs to be original since companies and teams are different.

  • General suitability for the role - Again this should differ from place to place

  • Core: Subject mater questions that focus on the main area of the job - This is scientific and does not need to be original since it is scientific.

  • Day-to-Day work suitability: These are questions about subjects you expect the candidate to do on a daily basis. It may be scientific, but it better be from the real job he is expected to do. This part would vary from place to place.

NoChance
  • 12,412
  • 1
  • 22
  • 39