Search This Blog


Saturday, February 9, 2008

Unit Testing, What a Pain!

Phil Haack told some of his pains in dealing with unit testing in the post Tell Me Your Unit Testing Pains. Here I would like to share some experience of my own.

I was doing desktop applications, CAD software like AutoCAD, so some of my experience may be different from those who do web applications. Unlike Phil Haack, I didn't mess around with ASP.NET, so I didn't have any issues in mocking the Framework. We needed to develop our own Framework on top of the .Net Framework. In retro respect I wished that there was a Framework, because at least an existing Framework would have been designed by many others who knew well about software development issues. Some of the unit testing pains, or should I say, a majority of the unit testing pains had to do with an improper design of our own framework.

I wasn't even thinking of unit testing when I first started my programming career. No one in my company practiced it, and even the most enlightened ones had had only some vague idea about it. We thought that unit testing was limited to methods or classes that were easy to test or had well-defined input/output. Static methods were obviously good candidates for unit testing. But Object Oriented Classes when all different types of instances interact together? No, not for unit testing.

And so I blithely developed my code, there were just pages and pages of these code. In the course of software development I cut corners, copy pasting similar logics, modifying method parameter calls without much thought, lumping different sets of data into a single class for convenient sake, premature optimization and practiced ( or committed?) other numerous software anti-patterns.

So in the end, at the maintenance stage, I looked at my code in awe. There simply was no way I could modify the code anymore without making subtle breaking changes. The code was highly coupled, lowly cohesive: exhibit A of How Not to Develop Software.

And I started to think of unit testing. But how to test spaghetti code that wasn't written with properly object oriented techniques and testability in mind?

My old code exhibited the following characteristics:
  • Free use of Singleton. In essence, you won't know what a class depended on unless you took a magnifying glass and took a good look of what was contained inside.
  • No separation of concern. Since we were not doing database, so we didn't use relational database such as SQL server etc. We used binary file to store the application data. Guess what, the logic of data access was tightly intertwined with the application logics, and the application logics were sprinkled in the midst of GUI code, right in the same class as the Form class and in the event delegate methods. Oh dear, try to test code that asked you to OK a dialog box out of nowhere? There was simply no way to get automated unit test.
  • Complicated dependencies between methods and classes. To properly test a method I needed to setup a lot of input objects that were difficult to setup. To make things worse, sometimes the whole purpose of setup input objects was to get only a single value or property from those objects. When you needed to write thousands lines of setup code just to test a single short method, you would lose all the interest in unit testing. There was simply no hope to test each method in isolation.
  • Don't know what to test! Yes, this was one of the hurdles I faced. As mentioned above, I was doing CAD software, the output could only be properly verified by looking at the screen and counting the objects, checking the colors etc. All the screen outputs were fuzzy; if they looked good, they were correct, if not, wrong. How to translate those requirements to precise logics and conditions?
So I concluded that the code was simply not testable and I threw away the existing code and wrote from scratch. The code is now in better form and there were some unit tests covering them.

Life had been better for me since then.

Follow up post:
  1. Unit Testing, What a Pain!(2)
  2. How to Evangelize Unit Testing


paulo said...

You're doing it wrong. If there is high coupling, unit testing helps you recognize it and is an incentive to fix it. If setup is complicated, it means you're not using factory classes that could simplify your main code immensely. If you're bitching about dialog boxes, you're probably unit testing at the wrong level of the software. I don't really know anything but this should be ob

Casper Bang said...

I have to admit of experiencing much the same when unit testing "real" applications (not library functions). Perhaps it's simply a matter of limited unit testing experiences and lack of tool knowledge (JMockit etc.).

Anonymous said...

I have found some ideas from "Working Effectively with Legacy Code" by Michael Feathers useful to untangle spaghetti and get things under test that I didn't think possible before.

Oleg Cherkasov said...

Don't know what to test!

Is the most important of all. If you know where it fails then you might not need UT at all! Of course regression testing is important but you may design it the way it would be stable regardless the way related functionality got updated.

Anonymous said...

Yeah, I agree. I wouldn't even attempt to do unit testing on an existing application. And I wouldn't do anything but test driven development if the goal was to have unit tests.

Unit/integration/etc testing is a development style, not something you tack on later. It influences everything about how you design your code. When doing TDD, all your code is testable, because it has to be. You simply write applications differently.

When doing TDD, I rarely debug, I rarely even run the app; I just watch the automatic tests happening as I code. When not doing TDD, I more program by debugging, I write some code without thinking too much, then debug to see if it works; it's that kind of process.

Personally I think that unit testing and TDD are one of the main reasons that dynamic languages are becoming more popular. If your goal is to have complete code coverage with tests, then a dynamic language starts to have huge benefits. And since you're completely covering your code, its drawbacks are minimized. Stubbing and mocking in a dynamic language are child's play, in a static language, painful.

If you're not doing unit tests, a dynamic language looses many of its benefits.

As for what to test, the answer is simple: everything.

Troy DeMonbreun said...

You might find a better experience with TSD:
Test Supported Development (TSD) is NOT Test Driven Development (TDD)