F
Frans Bouma [C# MVP]
David said:Unit tests are all well and good, but they do not solve all problems, and
there are many types of testing that unit testing does not begin to
address. Sometimes all that the unit test does is validate your own
misconceptions about the system. There are integration tests and systems
test that need to be performed and these cannot be performed in isolation.
first: I'm not a native english speaker, so if my sentences are formulated
wrong/badly, I'm sorry, but it's not my first language.
second 'you' and 'your' in my sentences are mostly used as a 'general' term,
I also could have used 'one'. Dutch has a word for this, English doesn't
afaik.
true, if your tests are flawed, it's not going to bring you anything, better
yet: it's worse than not having them.
There are also tests that require a different environment then is present
during the unit test, and duplicating this environment is extremely
difficult. And just so you know, we have hundreds of unit tests built into
the project, and they are all run as part of our automated build process.
ok, but I didn't know that, still you posted a problem as an argument in
favor of E&C. So all I read was: "I have this 10 minute period" (I admit, I
misread the sentence, it was compile + setup time, however it doesn't matter
much for the argument IMHO) and believe me, you're not the first with that
argument in favor of E&C. (this subject has been beaten to death in the
whidbey newsgroups) Most of the time when someone came with that argument, it
turned out their methods of software development in general weren't efficient
enough.
I always come with the Operating System development argument: it takes 7 to
8 hours to build windows XP if I'm not mistaken. No developer can test
his/her changes with such a long build cycle, so they use different
approaches to the problem: it's fixing the problem by the root of the cause,
not with a fix for the sympthom.
And to your original complaint...yes, the team got together and discussed
the situation and we decided that it wasn't worth the downside to break up
the solution into a number of smaller solutions; we decided to live with
the long edit/compile/debug cycles and we would look at doing it
differently on the next project. There are problems when you break up a
big solution into numerous small solutions.
It wasn't a complaint, just an advice. Remember: I don't have a problem, you
and your team have. So it's not me who complaints about something, you did
(and rightfully so, 10 minutes compile-setup time to get something tested is
unworkable).
There are problems to chop up a big solution into smaller ones, however I'm
not sure if those would be bigger than the big problem you have now. It will
only get worse in the future: what if a bug shows up when the application is
in production? That would be horrible to debug. E&C will not help you with a
lot of bugs either, as what it can do for you at runtime is limited (i.e.:
you can't add/alter all possible constructs, just a small subset)
You keep throwing the term "you" around - that makes it personal, and for
no good reason.
Why is that a bad thing? You posted the problem, so why should I address it
to someone else? Don't think I belittled you or anything, we see a lot of 'I
have a big solution and vs.net falls apart' problems here, so you're not
alone. But again, I just wanted to help. If you feel offended by me helping
you, I'll step back and leave you alone and your big project.
You assumed that the 10 minutes is solely because of the
compile time - I never said it was solely due to compiling. And you are
also assuming that this was "the" problem that I thought E&C would fix. All
I said it would help with that particular irritation.
... and I said it might not help you in that.
Then you are fortunate.
... or perhaps I organize better?

projects?
Unit tests are great when the functionality is entirely self-contained, and
IMO its usefullness diminishes as the amount of interaction with other
components of the system increases. It also decreases in usefulness as the
complexity of the configuration, setup, or complexity of the mock objects
increases. All code has bugs, including the test code. At some point the
complexity of the test negates the usefulness of the results as the
reliability of the result degrades.
There are other examples where unit tests aren't useful.
sure, however I used them as an argument to test routines which contain
bugs. You see, it starts with silly things like pre-post conditions. If a
given routine has a given set of pre/post conditions and these succeed, your
routine does what it should do and you can move on. This can easily be done
with unittests. The fun thing is: if all pre/post conditions of all routines
are tested, (even by hand, doesn't matter) your system is theoretically
correct. Then it comes down to: is the design correct and does it fail
because of unforeseen circumstances. At the same time you then also enter the
area where E&C is likely not going to help. I mean: you can attach a debugger
at any time to a running system, but also you can add trace listeners to a
running system. Monitoring is half the debugging.
My argument was: too little is done to prevent routines which are buggy
(i.e.: their pre/post conditions aren't met) to enter a system, which later
on causes problems.
Um, errr, I'm sure you really believe you mean well, but it comes across as
condescending and nasty.
I don't see why, I just try to help you with a problem. You apparently see
that as condescending, which I find odd, because I'm just trying to help and
what do I win with condescending you? I invest time to answer YOU. Believe
me, I have a lot of work on my plate every day, so I can spend my time better
than simply condescending people I don't know.
I'm sure you have something of value to contribute but the way you word
things and the approach you take is needlessly confrontational. I've had
many discussions up here and I've never had this experience with anyone
else.
Perhaps you MADE it personal and you assumed I wrote it as if I knew it all
and am here to tell you how it's done because you assumed I thought you were
stupid. David, if I think someone is stupid I don't even bother replying.
You're posts here are. You made an enormous number of assumptions about the
solution, the projects, why it took 10 minutes to get back to a particular
point in a debugging session, if it had unit tests, and the environment it
was running in, my background, and engineering practices in general.
Well, there wasn't a lot of info, was there, just the problem, which was
used as an argument for E&C. I didn't say anything about your background nor
engineering practises, I just used a general metaphore to explain why I think
in general people should pay more attention to common practises than to fixes
for a symphtom: fix the cause, not the result of the problem.
As I said earlier, 'you' can mean you David but also the people in general,
afaik. Again, I'm not a native english speaker. But I stop now, as I don't
have time to fight a silly battle over what I meant and wrote and how my tone
was and how I apparently condescend people, if all I want to do is trying to
help.
Frans.