Recently a potential client asked me to take a coding test on Codility.  (Well, they asked "us", not knowing that "us" was me :)  I took the test and failed miserably.  Herewith, lessons learned...

At right: the Codility interface ... (click to enbiggen)

My first mistake was taking the whole thing lightly.  I'm a great C++ programmer (if I say so myself) so I was confident in my ability to code.  However, there is more to passing a Codility test than writing good/great code.  1) you have to understand the interface, 2) you have to code an efficient solution, as opposed to a simple one that works, and 3) you have to do it in a limited amount of time.  Oh, and 4) you have to pay attention to boundary conditions.

1) To understand the interface, Codility recommend you take their sample test.  This is good advice which I did not take.  The interface is just fine, but it does take a few minutes to acclimate to it, especially the means of entering and running test cases.  In particular, they allow you to run any test cases you want, but they do not tell you whether the tests get the right answer, only whether they compile and run.  This isn't immediately obvious, and it's important.

2) Codility specify the O-notation of the expected best solutions.  Most of their algorithmic problems are pretty simple, but coming up with a most efficient solution may require tricks.  It's a mistake to go after a tricky / efficient solution first, however, as I found to my chagrin.  As I note below, code a simple solution that works first, then develop optimizations.

3) You are typically given 30 minutes to complete a test (I was given 90 minutes to complete three).  This is plenty of time to code and test a simple algorithm.  However, it is not plenty of time if you spend a lot of it learning the interface and investigating interesting optimal solutions.  Plan your time wisely.  Out of 30 minutes I suggest 10 minutes for a bolt dumb solution, 5 minutes creating a wide variety of test cases, and 15 minutes for optimization.

4) In many of their tests (I have now taken six) Codility specify boundaries like "integer in the range -2,147,483,647 ... +2,147,483,647.  You will recognize these are the max values for a 32-bit int.  They will give you these values in a test case.  If you want to add 1 to such an int, you'll need a long long.  This sort of stuff doesn't often come up in everyday coding, but you need to be ready for it.

My suggestions for anyone taking a Codility -type test:

• Take a sample test or two first, to get used to the interface.  This post discusses their sample problem (take that one first) and has links to two others.
• Develop your solutions in your favorite IDE, and then copy-and-paste them into Codility.  Really really.  At first you will think "this interface is fine, no worries" (as I did), but when you're pressed for time and want to run many test cases, you'll be happy to have your own IDE.  Mine is Visual Studio, yours might be Xcode or Eclipse, but use it.
• For each problem, develop the absolute most bolt-dumb simple solution first.  Make sure it works (with "printf" debugging, or whatever), and then use it as a check against your more complicated more efficient solutions.  This does two things for you: 1) if you run out of time, you have a solution that works, 2) having an instant test for any would-be more-efficient solution is most helpful.  You can run though many, many possible optimizations rapidly once you have an automated test.

In addition to Codility, other sites which offer these types of tests include TestDome and HackerRank.  They're actually a lot of fun once you get into the mode of taking them.

A final comment: In Googling I found quite a few people online have commented that this type of test is a bad measure of programming ability.  Okay, I agree it isn't perfect, and of course you will want other data points before hiring someone.  But it's a pretty good screen; ideally a good programmer would pass such a test, easily (*ahem*), and the fact that they've taken the time to do so is an indication of their sincere interest in your opportunity.

Cheers, and happy testing!