xUnit and .NET 4.0

Trying to run xUnit tests compiled against the 4.0 framework with the visual tool (xunit.gui.x86) I was welcomed by an error message telling me that the target .dll was compiled a more recent runtime than the one in use. After some googling I was able to find a fix for the console runner (xunit.console.x86), which consists in adding the following config section to the XML configuration file (xunit.console.x86.exe.config):

 
<startup>
    <supportedRuntime version="v4.0.30319" />
</startup>

(your .NET framework version may vary, take a look at your %systemroot%\Microsoft.NET\Framework for more info)

To make the fix work for the GUI runner you can just make a copy the modified console application’s configuration file and rename it as xunit.gui.x86.exe.config

Advertisements

Unit testing and API design

I have encountered the following situation more than once while working on freightrain: suppose you have a feature in mind, like being able to declare regions (which are other viewmodels, constructed dynamically and plugged into the parent viewmodel’s view accordingly) inside your viewmodel just like this:

class MainViewModel < FreightViewModel

  region :list #:viewmodel defaults to :list
  region :detail, :viewmodel => :customer_detail

end

This has been implemented using a couple of dirty tricks and, of course, it’s all properly tested. The thing is that the tests have been written after the implementation, in a completely separated coding session.

Mind that I am not a very big fan of test driven development: more often than not, especially when the customer is neither trained nor willing to acknowledge the extra effort and the improved quality that comes with it, the costs of test first approach outweigh the benefits it generates. In this case, though, things are different: quality is crucial and I am more than happy to spend time making things better.

Still, I wasn’t really comfortable with writing the test before the actual code. I think that’s because I had a very clear design in mind: testing would just have made the code more complicated than it needs to be. On the downside, the tests written for that piece of code look nothing like the kind of tests that you usually get when doing things TDD style.

What I think I learned from the experience is:

  • While TDD generally helps your design, there are some cases where it gets you to a second best. If you are extremely sure that your design idea is sound then you should abandon testing, do some cowboy coding and when after you’re satisfied with results look back and test everything. The obvious trap is to ditch testing indefinitely: don’t do that.
  • Application design is very different from API design and you have to act (and test) accordingly. Testing the how doesn’t feel that wrong at this level.
  • Sometimes it’s easier to solve a problem with an if or two than to rearrange things: it’s a bad idea and you will see it by the time you test. While ifs are tolerable at the application level they’re a real pain when working at a lower level, and the better the surrounding design the more they hurt. Don’t ignore the warnings.

Unit testing with ruby (part 3)

Here follows a dummy controller scenario :

While testing this method is certainly possible, given that both @view and @service are injected at some point before the callback is fired, tests are not going to be lean :

Mock objects are a necessary evil: when there’s no output to check and no clean way to access the modified state you can check for the correct calls to be made on your mocks. Sure, it looks better than no testing at all, but the price to pay is pretty high:

  • Verifying expectations means testing how and not what, which is what you really care about.
  • The tests become much less valuable since they are almost useless as documentation: it takes much less to go straight into the code than to understand what’s going on from reading an integration test.
  • The majority of the test code will be about setting up the environment. In the example the only 2 lines we care about are the one where the expectation is set and the last one, the actual method call on the controller. All the remaining code is just to allow the method to execute.
  • By the fact that you’re testing how mocking basically means duplication of your under-test logic. Adding insult to injury it also makes your tests extremely frail, because even a single change in your logic will reflect into lot of broken tests due to setups not working anymore.

When mocking becomes a source of too much hassle it’s often useful to stop testing and reconsider the code under test: there’s a good chance something ‘s wrong with the underlying design, much likely too much work is done by a single actor, usually the controller (or presenter, viewmodel, you pick it). For instance, assuming that the view mechanics are a given and cannot be modified, the example code would be a little better if refactored to be like this:

Controller methods should be a maximum of three line long, with no ifs. No kidding! I also drop the testing completely on controllers if i can get methods that simple: they add no value and are a pain to keep green, especially on the first stages of development.
In the end i think it’s important to remember that mocking (and unit testing, agile and what else) is just a tool and by the moment it gets in your way you should carefully reconsider what the benefits are: methodology is always out there looking for a chance to strike, don’t let your guard off 🙂

Unit testing with Ruby (part 2)

In ruby, you can do this:

Class methods mocking

Class methods mocking

Hum. Static’s bad, don’t do static, mkay? Well, not necessarily. The fact that you can test class methods like this makes them much more usable than they are in C#, where using static all over the place brings your design to the dumpster pretty quickly. Still, the use of class methods usually covers for underlying design flaws, so it’s a technical debt and you should evaulate what is convenient to do case by case.

The above snippet is powered by mocha, a mocking framework for ruby. It has a JMock-like syntax and it provides you all the usual tools.

mock()

creates a mock object. You can set expectations on a mock object with

mock_object.expects(:method_name)

and stubs with

mock_object.stubs(:method_name)

Expectations are verified at the end of the test: if you’ve set an expectation for :method_name and that method isn’t called on the mock object before the end of the test, that test will fail (no need to VerifyAllExpectations or such). Instead, stubs do not fail the test if not called.
So, in the following example:
mock_example

if we remove the object.second call from the method run only the second test fill fail.

As seen before, you can set expectations and stubs not just on mocks but on any object, even class objects. Abusing of this feature, as to say in object = Object.new; object.expects(:method) ,  is probably not wise but as shown in the first snippet this allows testing of usually not testable situations.

You can force constraints on expectations using with(param_value) and you can return values with returns(value). For example, the following code:

mock_object = mock()
mock_object.expects(:get_capital).with('sweden').returns('stockholm')

Sets an expectation on mock_object for the method get_capital, the expectation is satisfied only if get_capital is called with the string 'sweden' and, if called that way, returns the string 'stockholm'. Of course you don’t need both clauses to be present togheter, mock_object.expects(:get_capital).with('sweden') and mock_object.expects(:get_capital).returns('stockholm') work just as expected. If you don’t set a return value for any given expectation/stub it will return nil.

stub_everything() returns a mock object that stubs every possible method call. The return value will always be nil.

More on mocha will be found on its documentation page.

Unit testing with Ruby (part 1)

Not having the compiler on your side makes life difficult. Example:

C#

public int Multiply(int firstFactor,int secondFactor)
{
      return firstFactor*secondFactor;
}

Ruby

def multiply(first_factor, second_factor)
   return first_factor*second_factor
end

What these methods are supposed to do is pretty straightforward. What they actually do, maybe not so.  What if you were to test both methods? What\how many tests would you write to make sure these methods behave as you expect?

Let’s take a look at the C# method. It takes two parameters (both int, so value types) and returns an integer. Relying on cyclomatic complexity to determine, for a given method, how many tests must be written is often uncorrect but let’s assume that it’s okay for this case, so one test that verifies the result for a given input might be enough.

What about the Ruby method? It takes two parameters and returns something. And the two parameters might be pretty much anything. How about testing this one? There’s no IFs so we might say that one test is still fine, or you might want to check parameters for correctness, or maybe make sure that who calls your method do it the way it’s supposed to. Either options (except the first but seriously, you’re not going for that one) requires extra effort in comparison to the C# approach.

About mocking:

C#

public Controller(IView view, IModel model)
{
      //some stuff happening here
}

And Ruby

def initialize(view, model)
      #some stuff happening here
end

Mocking view and model in C# requires some degree of IView = repo.CreateMock<IView>(). This relies on the IView interface, and you’ve got nothing like that in Ruby. You just mock methods instead. Mocha syntax:

def test_refresh_always_callRefreshOnView
   view = Mock()
   view.expects(:refresh)
   controller = Controller.new(view,Stub())
   controller.refresh
end

No need of VerifyAllExpectations() or such: this fails automatically if refresh is not invoked on the view object.

A similar problem arises when using IoC containers: you’ve got no interface to resolve. More about this later.