Jeff Atwood over at Coding Horror recently posted 'How to Hire a Programmer' where he shared some techniques he's seen work when hiring developers in the past. Most of the post got a thumbs-up from me, but his section 4, "Do a detailed, structured phone screen" was ludicrious:-
- A bit of on-the-fly coding. "Find the largest int value in an int array." or "Write a function to reverse a string." or "Write function to compute Nth fibonacci number." or "Print out the grade-school multiplication table up to 12x12.";
- Some basic design. "Design a representation to model HTML." or "Design a deck of cards that can be used for different card game applications. " or "Model the Animal kingdom as a class system, for use in a Virtual Zoo program. ";
- Scripting and regular expressions. "Last year my team had to remove all the phone numbers from 50,000 Amazon web page templates, since many of the numbers were no longer in service, and we also wanted to route all customer contacts through a single page. Let's say you're on my team, and we have to identify the pages having probable U.S. phone numbers in them. To simplify the problem slightly, assume we have 50,000 HTML files in a Unix directory tree, under a directory called "/website". We have 2 days to get a list of file paths to the editorial staff. You need to give me a list of the .html files in this directory tree that appear to contain phone numbers in the following two formats: (xxx) xxx-xxxx and xxx-xxx-xxxx. How would you solve this problem? Keep in mind our team is on a short (2-day) timeline."
- Data structures. "When would you use a hashtable versus an array?" or "When would you use a linked list vs. a vector?" or "Can you implement a Map with a tree? What about with a list?" or "How do you print out the nodes of a tree in level-order (i.e. first level, then 2nd level, then 3rd level, etc.)";
- Bits and bytes. "Why do programmers think asking if Oct 31 and Dec 25 are the same day is funny?" or "Describe a function that takes an int value, and returns true if the bit pattern of that int value is the same if you reverse it (i.e. it's a palindrome); i.e.
boolean isPalindrome(int x) "
The post incensed some of the Coding Horror readers and I must agree with a number of them. When a bricklayer goes for a job with a bricklaying firm, is he asked to build a wall and describe the sand to cement mixture ratio? Doubtful. I'm not overly adverse to some of the methods picked out by Jeff, but I am concerned about how employers may read his blog, misunderstand his point (as did many) and introduce these as their tests.
When a prospective employer asks their candidate on the telephone that they wish to remove 50,000 telephone numbers from some static HTML pages, what's the candidate's preferred method, what are they expecting for an answer? Will regex or grep suffice? Do they want an example? "(\([0-9]{3}\)\s|[^0-9][0-9]{3}-)[0-9]{3}-[0-9]{4}\b"? Or do they want a clever candidate, one that thinks about the requirements? Even as someone who knows the answer, there doesn't seem to be a correct answer! Far too much is left to interpretation.
These questions aren't really testing a candidate and their programming ability at all. The test merely confirms if they are good or bad at telephone interviews and technical tests. A programmer may not know that 2^16 is 65536 (despite it being an important number), but may know that:-
short x = z + y;
...will throw a compilation error because z + y evaluates to an int by default. The correct way is to cast:-
short x = (short)(z + y);
Out of the two answers, which is the more useful to know? When, whilst working on commercial projects, have you ever needed to know that 2^16 is 65536 anyway?
One of the comments to Jeff's posts by Jorge summed it up perfectly:-
"...The first couple of interviews I went to I did very badly in the technical test. The interviewer thought that I wouldn't be up to the standard they were after. The 3 or 4th I did much better. The 5th and 6th I blitzed and got some very nice job offers!
OK, I may have felt the pressure a bit after bombing on the first two, so I have been doing some extra study. Did my technical skills really develop so rapidly over the past 2 months, or did my interview skills develop? Did the first interviewers just miss a good opportunity... or were the later interviewers duped?
The fact is, it's easy to get good at interviews after a couple of times. You hear the same questions every time, so you can go away and cover the holes in your knowledge..."
The UK development job-market these days are Monday to Friday 9 to 5 dev slogs, working on mundane applications built on some crappy code-heap-of-a-platform that you'll be expected to deal with. Often developing for a faceless, bland company with no sense of creativity or adventure. The majority of us don't get to work for Google, Stack Overflow or Facebook. So, if these mediocre dev houses come knocking on my door and decide to make me jump through all these hoops, I don't want your job.
If I was running an interview, I wouldn't have a paper test or even a practical one. I'd have some basic questions where the answer wasn't important. I'd just want to see how the candidate goes through the process of the solving the problem. Consider "ask the duck"...:-
Bob pointed into a corner of the office. "Over there," he said, "is a stuffed duck. I want you to ask that duck your question."
I looked at the duck. It was, in fact, stuffed, and very dead. Even if it had not been dead, it probably would not have been a good source of design information. I looked at Bob. Bob was dead serious. He was also my superior, and I wanted to keep my job.
I awkwardly went to stand next to the duck and bent my head, as if in prayer, to commune with this duck. "What," Bob demanded, "are you doing?"
"I'm asking my question of the duck," I said.
One of Bob's superintendants was in his office. He was grinning like a bastard around his toothpick. "Andy," Bob said, "I don't want you to pray to the duck. I want you to ASK THE DUCK YOUR QUESTION."
I licked my lips. "Out loud?" I said.
"Out loud," Bob said firmly.
I cleared my throat. "Duck," I began.
"Its name is Bob Junior," Bob's superintendant supplied. I shot him a dirty look.
"Duck," I continued, "I want to know, when you use a strap hanger, what keeps the sprinkler pipe from jumping out of the strap when the head discharges, causing the pipe to..."
In the middle of asking the duck my question, the answer hit me. The strap hanger is suspended from the structure above by a length of all-thread rod. If the pipe-fitter cuts the all-thread rod such that it butts up against the top of the pipe, it essentially will hold the pipe in the hanger and keep it from bucking.
I turned to look at Bob. Bob was nodding. "You know, don't you," he said.
"You run the all-thread rod to the top of the pipe," I said.
"That's right," said Bob. "Next time you have a question, I want you to come in here and ask the duck, not me. Ask it out loud. If you still don't know the answer, then you can ask me."
"Okay," I said, and got back to work.
The very process of answering the question reveals the answer. It is this "aloud thinking" you want to see from a candidate. This implies an ability to problem solve themselves and at the same time, learn the answer and logic - thereby making better programmers.
Some time ago I was asked to undertake a technical test for a role and the only convenient time for both parties happened to be on my commute. I refused to undertake any technical test whilst driving, but would be happy to discuss technical subjects to show proficency. This was thoroughly frowned upon and questioned. I promptly decided the role wasn't for me, another mediocre dev shop who treat programmers like crap - do you need a programmer who can drive and program at the same time?
Technical tests in the interview process will find you candidates that are good at technical tests and interviews, not necessarily good programmers.
Thursday, May 3, 2012 at 5:15PM 
