Wednesday, February 22, 2012

Unit Testing Using Generated Test Cases

This is a follow-up post to my super simple php unit test base class that I wrote in a bind for Facebook's HackerCup. In that post I provided a real test I created using my base class jphpunit which included a test using the problems sample input and output and another test that used a generator. Today I will share the generator I used to help me test my code against the full set of constraints of the problem. This or a similar pattern can be used in your tests, in fact I highly recommend PHPUnit.

I created an abstract class for my test case generator so that I can use the same test pattern for each of the Facebook problems. <?php /** * Generates input and output data for use in unit tests. */ abstract class fbbase_base_generator { private $answer = ''; private $test_cases; public function __construct($numTests = 20, $options = array()) { // Generate test cases for ($i = 1; $i <= $numTests; $i++) { $this->test_cases[$i] = $this->_createTestCase($options); } date_default_timezone_set('America/Los_Angeles'); } public function saveFile() { $filename = 'data/generated_'. date('Y-m-d_H-i-s') .'.txt'; $input = count($this->test_cases) ."\n"; foreach ($this->test_cases as $i => $test_case) { $input .= $test_case['input'] ."\n"; } file_put_contents($filename, $input); return $filename; } public function getCorrectAnswer() { if (!empty($this->answer)) { return $this->answer; } $answer = ''; foreach ($this->test_cases as $i => $test_case) { $answer .= 'Case #'. $i .': '. $test_case['output'] ."\n"; } $this->answer = trim($answer); return $this->answer; } /** * @param array Any options that you want your test * certain conditions are passed directly to this * function from the constructor. * @return array An array of the test case, should * have 'input' and 'output' key indexes. Input is * the line that will go in the input file, output * is the expected output after the "Case #{$i}:", * do not include leading or trailing whitespace * unless it is expected as part of the output, as * they will be added for you. */ abstract protected function _createTestCase($options); } This base class allows me to define the _createTestCase() method with the details of the problem at hand without having to worry about re-writing the code that will remain the same no matter what the problem is.

Here is an example of the class in use. <?php /** * Generates input and output data for use in unit tests. */ require_once('fbbase_base_generator.php'); class alphabet_soup_generator extends fbbase_base_generator { protected function _createTestCase($options) { $alphabet = 'BDFGIJLMNOQRSTVWXYZ'; // A partial alphabet so we don't // accidentally create more instances of HACKERCUP $soup = array(); // An array of all letters in our soup // Add letters for HACKERCUP $hacker_cups = rand(0,30); for ($i = 0; $i < $hacker_cups; $i++) { $soup[] = 'H'; $soup[] = 'A'; $soup[] = 'C'; $soup[] = 'K'; $soup[] = 'E'; $soup[] = 'R'; $soup[] = 'C'; $soup[] = 'U'; $soup[] = 'P'; } // Choose a partial set of our HACKERCUP letters to include $set = rand(0,5); $num_sets = rand(1,10); for ($i = 0; $i < $num_sets; $i++) { // Add letters from our partial alphabet $soup = array_merge($soup, str_split($alphabet)); // Add our partial set of HACKERCUP letters switch($set) { case 0: $soup[] = 'H'; $soup[] = 'A'; $soup[] = 'C'; $soup[] = 'K'; $soup[] = 'E'; $soup[] = 'R'; break; case 1: $soup[] = 'C'; $soup[] = 'U'; $soup[] = 'P'; break; case 2: $soup[] = 'H'; $soup[] = 'A'; $soup[] = 'C'; $soup[] = 'C'; $soup[] = 'U'; $soup[] = 'P'; break; case 3: $soup[] = 'K'; $soup[] = 'E'; $soup[] = 'R'; $soup[] = 'U'; $soup[] = 'P'; break; case 4: $soup[] = 'H'; $soup[] = 'C'; $soup[] = 'R'; $soup[] = 'C'; $soup[] = 'U'; break; case 5: $soup[] = 'H'; $soup[] = 'A'; $soup[] = 'C'; $soup[] = 'K'; $soup[] = 'R'; $soup[] = 'U'; $soup[] = 'P'; break; default: $soup[] = 'H'; $soup[] = 'A'; $soup[] = 'R'; $soup[] = 'K'; $soup[] = 'E'; $soup[] = 'P'; } } // Jumble our soup and then implode it to words shuffle($soup); $input = ''; foreach ($soup as $letter) { $input .= $letter; // Randomly add a space, approximately every 8 characters if (rand(0, 8) === 0) { $input .= ' '; } } return array('input' => $input, 'output' => $hacker_cups); } } In this case the problem was that you are given a list of phrases that are written in alphabet soup, and need to figure out how many times you can write HACKERCUP using only the letters from that phrase. Since it doesn't matter what the phrase is (or that is made up of real words), the test cases generated are kept simple. You also want to make sure you add enough complexity into the test cases to ensure your code meets some of the edge cases. For instance, I added extra characters from the word HACKERCUP, but was careful not to inadverently generate a test case that would have more instances then would be expected.

Finally, here is how the generator can be used in a unit test. I'm using my jphpunit class, but it should work as-is in a PHPUnit test as well. <?php // Inside your unit test public function testGenerated() { $gen = new alphabet_soup_generator(); $fixture = $gen->getCorrectAnswer(); $input = $gen->saveFile(); $output = array(); exec('../alphabet_soup.php '. $input, $output); $this->assertEquals($fixture, implode("\n", $output)); }

Saturday, February 18, 2012

A Super Simple PHP class for Unit testing

Since I was introduced to unit testing and test-driven development, I have become a big fan. Since most of my projects to date have been primarily PHP, I prefer PHPUnit as my unit testing framework.

To me the best part of unit testing is that good unit tests are quick, automated confirmation that everything is working as expected. Using test-driven development I can write a test to ensure that given a certain input my future code will return a given output. I often find writing my tests first also helps define what I want my application to do, it is an outline of what the code should do.

Then as I write my code, I can run the test often to ensure that I don't get false positives and it will also point out any syntax errors or typos in my code.

Recently I found myself without my trusty testing tool and without internet at the time to download it. I was about to enter Facebook's HackerCup and needed to be sure my answers were correct before submitting them. So I ended up writing a quick, very simple unit testing base class for my submissions. I call it jphpunit because it is my version of a php unit tester and I based much of how it is used and the output it generates on my experience with PHPUnit.

It only has one class, jphpUnit, and one external method, assertEquals($expected, $actual, $message = NULL).

Here is a quick sample of one of the unit tests I wrote using it for my HackerCup submissions. When working on the problems, I would copy and save the example input and output files to run a basic test on.
<?php /** * alphabet_soup_test.php */ require_once('helpers/alphabet_soup_generator.php'); class alphabet_soup_test extends jphpunit { public function testExample() { $fixture = file_get_contents('data/example_output.txt'); $output = array(); exec('../alphabet_soup.php data/example_input.txt', $output); $this->assertEquals($fixture, implode("\n", $output)); } public function testGenerated() { $gen = new alphabet_soup_generator(); $fixture = $gen->getCorrectAnswer(); $input = $gen->saveFile(); $output = array(); exec('../alphabet_soup.php '. $input, $output); $this->assertEquals($fixture, implode("\n", $output)); } }
As you can see, I wrote a test that would generate other test cases based on the contraints of the problem to ensure my program met all contraints of the problem and not just the ones in the example. I will probably talk about this more in one of my next posts.

Sunday, February 5, 2012

PHP Increment Operator Bug? ... Nope!

So this is one of those things that I ran into recently that I never knew, that I probably should have.

First the problem. I was working on a project and had a snippet of code similar to this:
$a_counter = has_some_setting(); foreach ($arr as $key => $val) { if ($val == $some_condition) { $a_counter++; } } echo $a_counter;
I was expecting the output to be greater than 1, but for some reason it was always 1. To debug, I adjusted the code to see that my value should be being incremented.
$a_counter = has_some_setting(); foreach ($arr as $key => $val) { if ($val == $some_condition) { echo $val . ' == ' . $some_condition . '
'; $a_counter++; } } echo $a_counter;

Indeed it was, so why was $a_counter++ not working? Changing it to $a_counter += 1; worked fine.

At first I thought it was a PHP bug, and after doing a Google search, found a PHP Increment/Decrement bug report. Lo and behold, that is exactly what my problem was. My has_some_setting() function returns a BOOLEAN, not an INTEGER. And according to the PHP docs, this is not a bug.
Note: The increment/decrement operators do not affect boolean values. Decrementing NULL values has no effect too, but incrementing them results in 1.

So, the long and short of it is, if your increment/decrement operators aren't returning values you expect, make sure your variable is not a boolean. Also, it may be worth reviewing the Increment/Decrement operators on strings example.

Saturday, February 4, 2012

Nostalgia and Ramblings

It has been a while since my last blog post here. Re-reading my old posts makes me realize how little I knew then and how far I have come over the last few years. That is one of the things I enjoy most about being a web developer, it forces you to constantly learn and improve. I have always said that is one of my personal life-long goals; to never stop learning and improving.

I recently had a conversation with my mom about that too. She was saying that even as a little kid, I was always wanting to learn something new. So much so that she said they often couldn't keep up. As a father of two young girls, I'm amazed at how fast they learn and really how smart my girls are. Hearing some of the stories my mom was telling me about me at their age made me realize that this life-long goal of mine has really been ingrained in me from before I even realized it.

I find it very intriguing how smart children really are and how important enforcing the desire to learn is at such a young age. It seems like something we don't even realize most of the time. It also makes me think about conversations I've had with co-workers. There are so many times that we've worked with people that did not know something, and instead of trying to figure it out or learn, would ask somebody else. At my last job, (FYI, I've been working at Metal Toad Media since August), I was the person everybody asked. The thing that drove me nuts about it was that I often did not know the answer myself. I would use trial-and-error and/or Google and read to find an answer. That isn't to say people shouldn't ever ask, but it just seems that there are more and more people that don't have that same desire to learn. Even if they don't have that same passion, I can't seem to understand why they don't use the resources already available to them, and apparently I'm not the only one that feels that way. I'm not that mean-spirited to actually send somebody such a link, but I can see how some people might. I guess it just makes me see how important it is to instill and encourage my kids natural passion for learning and imagination.

Working at Metal Toad has been an eye-opener as well. My last job was so hectic and we were always under pressure to get things done "quick and dirty" and then "we can clean things up later". Later never came and while we were always told to do good work and only deliver something we could say we were proud of, it seemed to be an illusive goal. It is no wonder I grew tired of the way things were and grew to despise how I had gone from a company where I was able to learn and grow to a company that was behind the times with no plans to improve. Metal Toad has been such a contrast to that, it has literally been a culture-shock. Not only does Metal Toad encourage R&D time for each of its Developers, every week, but its pretty much a requirement. It allows Metal Toad to stay on the cutting-edge, and ensures that its Developers are the best in the business.

Along with R&D time, we are encouraged to blog regularly. In fact, at my first review, their only "complaint" was that I had only 1 blog post. The company holds blogging as a great way for developers to document what they have learned and over time it also helps establish the company and the individual as an expert in their field. Just like a doctor writes in a medical journal, a developer should blog.

This is something I have not been as good at, and in an effort to get better at it, I am going to start. At first, when reading my older posts, I was tempted to think that I would sound silly for "not knowing" and was discouraged. I'm realizing though, that it doesn't matter what I don't know now, it only matters that I will learn and get better. In fact, it will probably help me grow faster, and might, if I'm lucky, help somebody else.

So, expect more soon, both here and on my Metal Toad blog. This blog will probably be a little less formal (as you can probably see already), and will probably contain a few personal rants, definitely a few mistakes and a lot of learning.