Test Incest

Don’t you just love my cutesy rhyming post titles?

You can’t hate me, though, because I make sure most of them are entirely relevant. And this post is about the incestuous unit tests I write, and how they have bitten me.

Professionally, I have never been in the position to use a mock object framework like JMock or NMock. My mocks have been more like fake objects, since they were hand-written and always returned pre-cooked responses.

Since I had to hand-write these objects, I took what seemed to be a smart course of action.

Well, first I need an example. Let’s say I have a Controller, who talks to a Domain Service. The Domain Service talks to Domain Object A, Domain Object B, and their two respective repositories. Each repository talks to the Data Access Service, which finally talks to a SQL Server driver.

We need a picture. Well, I need one, anyway.

The way I write fake objects for this situation is to write two fake repositories:

Each fake implements the same interface that the real repositories implement. You know I’m a stickler for semantics. These are not mock objects because you do not set expectations on them. They are fake objects because they return pre-cooked responses. And these fake objects carry the particularly, as we’ll see shortly, insidious property of state.

If I only had to test retrieval of the domain objects, this would be no sweat. But when it comes time to test saving domain objects to their respective repositories, I make a colossal mistake.

“Hmmm. Some of the methods in Domain Service involve storing and retrieving various domain objects. In their tests, I’ll probably have to store a domain object, and then retrieve it back immediately thereafter and assert it is what I just stored. Why don’t I make each fake repository a Map behind the scenes?“

Pain. Fury. Frustration.

This is a huge two-pronged mistake. First off, I just made my fake repositories living, breathing applications in their own right. Now instead of one application to test, I have two!

How can this be? It’s just a Map. Yes, but if you’re in a language with pass-by-value semantics, the fake repositories do not behave like the real ones do because of aliasing.

If you use the fake repository to retrieve a domain object, make a change to it, and store it, the changes are visible to the repository immediately because the object you are updating is the same object the fake repository has a reference to.

If you want to do this correctly, you’ll need to create deep copies of the domain objects before storing them in the Map. As I said, this just became its own application.

This is one problem of not using true mock objects. But this is merely a precursor of an even larger mistake. I have scimped on the mocking of external dependencies. Even if I use a true mock, I will only have mocked one of the mockable external dependencies. This causes my unit tests to be very brittle.

Let’s say I make a change to Domain Service in the course of normal Programming Excellence. I update the unit test for Domain Service. All is well. Then I re-run my entire suite and find a host of failures in the tests for Controller because Controller is dependent on Domain Service, which has changed.

This is to be expected. If component A depends on component B and you change component B, component A needs to be re-tested. Re-integration-tested. The unit tests should be isolated and not affected.

At work, I have written the majority of our unit tests like this, and so whenever changes are made, a slew of unit tests likely break. It has become an unstated sole duty of mine to go in and fix them, whether I made the change or not. I made that brittle bed, so it’s only fair that I sleep in it.

Let this be a warning to you. Don’t be lazy with your mock objects. Invest the time upfront. Mock correctly and aggressively. The payoff will be clear down the road.

Announcer: You’re reading the EIP web-ring.