Can you cover me? Looking for a .NET coverage tool
Recently at the Day Job, my boss's boss has been on a "code confidence" kick. We've always done various levels of automated and manual unit, integration, issue, system, and regression testing, but he's looking to improve the process. Part of this push involves getting better at measuring which tests exercise what parts of the code. We want to know this for the usual reasons: we can identify gaps in our testing, or more likely find opportunities to cover some areas earlier in the testing cycle. It'd be nice to know that a particularly critical section of code has been adequately exercised by the per-build unit tests, without having to wait for nightly integration testing or wait even longer for a human to get their grubby mitts on it.
To that end, I'm looking for a .NET coverage tool to dazzle us with tales of test runs. Over the next little while, I'll look at a few candidates, summarize my findings, and hopefully come up with a winner.
Considerations
Here are some factors that will influence me. Some of these may be negotiable, if a candidate really shines in other areas.- We'd like to see coverage information in our build reports, so the tool should run from the command line.
- It'd be easier to put the coverage info our our build reports if the coverage reports were in XML.
- I really prefer a product that has an auto-deploy, so it can be bundled with the source tree and new developers or build servers just work. You may remember the pains I went to to auto-deploy TypeMock Isolator.
- While I'm on the subject, one of our products uses Isolator as its mocking framework, so the coverage tool should be able to link with TypeMock Isolator.
- We have a web services layer, which will be exercised by unit tests, but if we could gather stats on the layer as it's being exercised by the client-facing portion, that would be gravy. To that end, it should be possible to cover IIS.
- When I used TestDriven.NET + NCover a few years ago, I enjoyed being able to quickly see what my tests covered. This isn't a requirement of our current initiative, but IDE integration would be a bonus.
- Price is a factor. Money's available, but why spend if you don't have to? Or at least, why not pay less for an equivalent product.
The Candidates
Googling has lead me to these candidates, which I'll be examining in the next little while:Update: I picked one.