Good devs are hard to find. There just aren’t that many of them out there and you have to go through a lot of people to find them. Because this can be a very time consuming process, I’m always looking for ways to become more efficient in identifying gems in a sea of mediocrity.

Standardized Testing to the Rescue

An appealing silver bullet to this problem is standardized tests. The basic idea is great: a candidate you’re interested in is given an online test in technologies of your choice. In a few hours, you get a nice score along with a helpful comparison to national averages.¬†Awesome, right? I mean, just think of how many hours of painful conversation with unqualified candidates you can save yourself!

Well, maybe. For the test to be useful, it has to be able to correlate candidate’s actual abilities to their score. Otherwise, much like various certifications, its utility is limited.¬†So, how well do these tests really work?

Testing the Tests

To answer my own question, I looked at a set of standardized tests from a well known online testing company (which shall remain nameless because I don’t want to get sued). I actually took a few tests on various technology topics (.NET, JavaScript, OO principles).

Overall, I found the tests to be largely inadequate. My primary complaint is that questions are quite shallow and make almost no attempt to test in-depth knowledge. For example, the JavaScript test didn’t ask any questions about key concepts of the language: prototypical inheritance, functions as first class citizens, closures, etc. The OO test had virtually no questions on SOLID principles and just one on design patterns.

On a separate but related note, a surprising number of questions was of poor quality. As in, too many questions were oddly phrased and/or had confusing /conflicting instructions. I don’t know if this was a problem with the vendor or the specific tests I chose to take, but it was certainly disappointing.

Bottom Line

It’s hard to test devs. More specifically, it’s hard to come up with good questions to ask them. Even more specifically, it’s hard to come up with questions that test understanding.

As tempting as it is to do, it’s just not very helpful to be asking a bunch of syntax questions. After all, you’re not looking for trivia buffs, you’re looking for people who have mastery of concepts behind the syntax.

Also, test questions, if at all useful, should probably not be used as the sole qualification criteria. You should seriously consider having devs write code as part of the interview process. You’d be surprised how much you can tell about a developer from their code submission.

You may also like:

Did you love / hate / were unmoved by this post?
Then show your support / disgust / indifference by following me on Twitter!

Comments are closed.