diferent results in debugger

[colin]

Hi,
Ive got a difference in results depending on wether I run my app
in the debugger, or run it seperatly (or with <ctrl-f5>)

the results in the debugger seem to be more correct, although
the app isnt finished and theres still a lot of things it gets wrong.

the processing is so complex its hard to determine where the difference can
be occuring,
its a visual display of 3d models, and theres a couple of surfaces
wich arent the same.

theres a lot of complex polyhderon to polyhedron intersection processing.
takes about 10 seconds of cpu time, simpler runs dont show the difference.

If I run the debug or the release version the results are the same,
it just depends if the debugger is attatched or not.

I thought the debugger made no diference,
certianly I cant see how it can make a difference to my results ?

I'm confused, anyone got any idea where to start looking ?
ive not used any unsafe code, but it reminds me of c++ with pointers
pointing to random places ... c# wouldnt allow this sort of thing would it?

thanks
Colin =^.^=

 

[Michael Nemtsev]

did u try to log intermediate results to compate them afterwards?

btw, using unit tests will mitigate this task

 

[Peter Duniho]

Ive got a difference in results depending on wether I run my app
in the debugger, or run it seperatly (or with <ctrl-f5>)

the results in the debugger seem to be more correct, although
the app isnt finished and theres still a lot of things it gets wrong.
[...]
I thought the debugger made no diference,
certianly I cant see how it can make a difference to my results ?

I'm confused, anyone got any idea where to start looking ?

Without a concise-but-complete sample of code that reliably reproduces the
problem, it's very difficult to offer anything useful.

That said, _if_ you have multiple threads operating in your code, then my
first suspicion would be that you have failed to properly synchronize your
threads somehow.

Usually it's because one is actually stepping through code in the
debugger. This often has the effect of adding some synchronization
inadvertently, or at least changing the timing enough to avoid problems.
But it's conceivable that even if you're not stepping through code in the
debugger, the presence of the debugger could change the timing of the code
enough to change the outcome.

If you don't have multiple threads, then I wouldn't even hazard a guess as
to what might be the problem. In either case, you should really do some
work to come up with a concise-but-complete sample of code that reliably
reproduces the problem.

I realize your application is complex and it may be very difficult to
narrow things down. But that doesn't change the importance of doing so.
If anything, the more complex the application, the more methodical and
careful you need to be in trying to track down difficult problems like
this.

Pete

 

[Chris Mullins [MVP - C#]]

A very common problem is doing real work in properties.

When you inspect a variable in the debugger, it may go through the property
accessor. If so, any work you do in the accessor will happen. This means if
you're updating a log, or tracking a count, inspecting a variable in the
debugger will cause all sorts of problems.

Is you app using threads and parallelism? If so, you may have a bunch of
concurrency bugs...

 

[colin]

Ive got a difference in results depending on wether I run my app
in the debugger, or run it seperatly (or with <ctrl-f5>)

the results in the debugger seem to be more correct, although
the app isnt finished and theres still a lot of things it gets wrong.
[...]
I thought the debugger made no diference,
certianly I cant see how it can make a difference to my results ?

I'm confused, anyone got any idea where to start looking ?

Without a concise-but-complete sample of code that reliably reproduces the
problem, it's very difficult to offer anything useful.

That said, _if_ you have multiple threads operating in your code, then my
first suspicion would be that you have failed to properly synchronize your
threads somehow.

Usually it's because one is actually stepping through code in the
debugger. This often has the effect of adding some synchronization
inadvertently, or at least changing the timing enough to avoid problems.
But it's conceivable that even if you're not stepping through code in the
debugger, the presence of the debugger could change the timing of the code
enough to change the outcome.

If you don't have multiple threads, then I wouldn't even hazard a guess as
to what might be the problem. In either case, you should really do some
work to come up with a concise-but-complete sample of code that reliably
reproduces the problem.

I realize your application is complex and it may be very difficult to
narrow things down. But that doesn't change the importance of doing so.
If anything, the more complex the application, the more methodical and
careful you need to be in trying to track down difficult problems like
this.

Pete

thanks for all the replies,
unfortunatly the nature of the problem makes it dificult to write test
routines wich help that much,
although ive written a fair amount of internal testing to ensure integrity
etc,
the real test is using it on real live 3d models, where ive found lots of
problems I hadnt anticipated.

the difficult problems to deal with are things like 3d points being very
close to 3d lines etc,
making decisions where slight rounding differences could lead to a different
outcome.

I have a very limited amount of thread interaction,
the form thread reads the file, and hands the result over to the xna 3d
window thread,
theres no interaction the other way.

the problem occurs just running with no stepping etc.

the problem is also very repeatable, so thread timing seems strange to cuase
this problem
although not unthinkable.

it also needs to chew on about 20mb of data, smaller input data doesnt seem
to have a noticable diference,

the data is displayed on a 3d view so its hard to do an exact comparison,
but I could dump the transformed 3d models into a file and compare them,
but then figuring out whats gone wrong seems difficult to say the least.
seems a lot of work, il try and look at the thread timing and
synchronisation.

I thought it was worth seeing first if there was some diference that may
have been more significant.

Colin =^.^=

 

[Peter Duniho]

[...]
the difficult problems to deal with are things like 3d points being very
close to 3d lines etc,
making decisions where slight rounding differences could lead to a
different outcome.

[...]
the data is displayed on a 3d view so its hard to do an exact comparison,
but I could dump the transformed 3d models into a file and compare them,
but then figuring out whats gone wrong seems difficult to say the least.

It will be tedious work. But this could in fact be the most fruitful
approach, especially given what else you've written.

seems a lot of work, il try and look at the thread timing and
synchronisation.

I thought it was worth seeing first if there was some diference that may
have been more significant.

Well, as you said, the debugger really should not affect program flow at
all.

I still wonder if there's some timing issue, but if not there's another
possibility I can think of: the debugger is somehow influencing the FPU
state, which consequently changes how your calculations come out. Someone
else posted a long while ago here about something they ran into where
creating a Direct3D device changed how floating point calculations were
done. It's not inconceivable that the debugger might somehow influence
this as well, depending on how the rest of your code works.

Unfortunately, regardless of the difficulty, this is unlikely to be solved
until you figure out a way to isolate the problem.

With respect to your comment about comparing the output data: as you
suggested, try to see if you can find some consistent relationship between
the outcome. Based on your description, it would be especially
interesting if you find that the calculation results are different for
every calculation, but you only have certain situations when it makes a
drastic different (for example, you've got some distance comparison in
which a threshold is used to determine the output, effectively quantizing
any differences in the output...tiny differences in a calculation wouldn't
show up visually in 3D data, but they would when you use the calculations
to turn things on or off, or connect or not connect things, etc.)

Pete

 

[colin]

[...]
the difficult problems to deal with are things like 3d points being very
close to 3d lines etc,
making decisions where slight rounding differences could lead to a
different outcome.

[...]
the data is displayed on a 3d view so its hard to do an exact comparison,
but I could dump the transformed 3d models into a file and compare them,
but then figuring out whats gone wrong seems difficult to say the least.

It will be tedious work. But this could in fact be the most fruitful
approach, especially given what else you've written.

seems a lot of work, il try and look at the thread timing and
synchronisation.

I thought it was worth seeing first if there was some diference that may
have been more significant.

Well, as you said, the debugger really should not affect program flow at
all.

I still wonder if there's some timing issue, but if not there's another
possibility I can think of: the debugger is somehow influencing the FPU
state, which consequently changes how your calculations come out. Someone
else posted a long while ago here about something they ran into where
creating a Direct3D device changed how floating point calculations were
done. It's not inconceivable that the debugger might somehow influence
this as well, depending on how the rest of your code works.

thanks I was thinking something like FPU state or something...
or maybe cuases some change in operating mode,
however the device is created at the start fairly early on,
reloading the file later produces the same results as the first time.

Unfortunately, regardless of the difficulty, this is unlikely to be solved
until you figure out a way to isolate the problem.

it might be a big task to examine a dump of the actual end result,
il look at other options first.

With respect to your comment about comparing the output data: as you
suggested, try to see if you can find some consistent relationship between
the outcome. Based on your description, it would be especially
interesting if you find that the calculation results are different for
every calculation, but you only have certain situations when it makes a
drastic different (for example, you've got some distance comparison in
which a threshold is used to determine the output, effectively quantizing
any differences in the output...tiny differences in a calculation wouldn't
show up visually in 3D data, but they would when you use the calculations
to turn things on or off, or connect or not connect things, etc.)

Pete

Ive gone as far as plotting the data either side of the point at wich I make
a decision,
about points being the same or different, some input data
points are either >5 units away or closer than 0.01
however some data has a range of points spread out,
I have my decision point between 0.05 and 0.5.

I do have a certain amount of debug information wich is written to the
console,
but ofc I cant see if thats changed when I run it without the debugger,
actually I wonder if that itself could be cuasing a differece ?
I did look to make sure I wasnt actually changing any thing inside
the actual WriteLine statement.
where does it go when theres no debugger?

il have to write it to a file to get more detailed debug info.
actually theres a good chance this might give me enough information
to go on.

thanks
Colin =^.^=

 

[Jon Skeet [C# MVP]]

thanks I was thinking something like FPU state or something...

That may very well be the case. You could certainly see very slightly
different results to FP operations depending on whether it's in debug
or release mode. For instance, sometimes an operation will be
calculated in 80 bits and remain in 80 bits if there's no reason to
truncate it to 64 - but in debug that optimization may not occur. In
particular if you start displaying results to the console, that could
certainly change what optimizations are applied.

However, you should only see this in edge cases, where either result
can be seen as acceptable.

 

[colin]

That may very well be the case. You could certainly see very slightly
different results to FP operations depending on whether it's in debug
or release mode. For instance, sometimes an operation will be
calculated in 80 bits and remain in 80 bits if there's no reason to
truncate it to 64 - but in debug that optimization may not occur. In
particular if you start displaying results to the console, that could
certainly change what optimizations are applied.

However, you should only see this in edge cases, where either result
can be seen as acceptable.

yeah but its identical in debug or release mode,
its only diferent if the debugger is 'attatched'
wether debug or release, wich is more strange ...

all my operations are done in float anyway as it uses the xna vector3
and related matrix operations wich again are all done with float.

I wish I could do them all in double actually ...

Colin =^.^=

 

[Jon Skeet [C# MVP]]

yeah but its identical in debug or release mode,
its only diferent if the debugger is 'attatched'
wether debug or release, wich is more strange ...

Not really - don't forget that in .NET most of the optimisation is
performed by the JIT compiler, not the C# compiler - and the JIT's
optimisations are heavily affected by whether or not a debugger is
attached.

all my operations are done in float anyway as it uses the xna vector3
and related matrix operations wich again are all done with float.

I wish I could do them all in double actually ...

You'll probably see *more* differences if you're using float instead of
double. This could well be the issue.

 

[colin]

Not really - don't forget that in .NET most of the optimisation is
performed by the JIT compiler, not the C# compiler - and the JIT's
optimisations are heavily affected by whether or not a debugger is
attached.

Aha I see thanks I didnt realise that,.. when I run the release version in
the debuger it
is a fair bit quicker, not to mention some lines obviously get optimised
out.

are some of the .net modules affected by this too?
or the Directx/Xna references ?

You'll probably see *more* differences if you're using float instead of
double. This could well be the issue.

so in the debugger will it be doing it at a highr precision ?
it seems to give more correct solution with the debugger attatched.

I gues il have to get some results out in the form of numbers soon,
but for now im trying to rewrite my PolyhedronIntersectPolyhedron routine
wich gets called 10million times ...
so im trying to make it faster by pre calculating and saving
as many many of the repeated calculations as possible.

Colin =^.^=

 

[Jon Skeet [C# MVP]]

Aha I see thanks I didnt realise that,.. when I run the release version in
the debuger it is a fair bit quicker, not to mention some lines obviously
get optimised out.

are some of the .net modules affected by this too?
or the Directx/Xna references ?

Well, there will be *some* more optimisation performed by the C#
compiler - but you'd presumably be using the same DirectX code either
way.

I have to say, I'm pretty much in the dark when it comes to DirectX -
it could be that there are oddities with what DirectX does when a
debugger is attached.

so in the debugger will it be doing it at a highr precision ?
it seems to give more correct solution with the debugger attatched.

Hmm... that's not what I'd expect, to be honest. I'd expect it the
other way round, with higher precision results in the more optimised
code due to it not doing a conversion in between.

I gues il have to get some results out in the form of numbers soon,
but for now im trying to rewrite my PolyhedronIntersectPolyhedron routine
wich gets called 10million times ...
so im trying to make it faster by pre calculating and saving
as many many of the repeated calculations as possible.

That makes sense. Does it have to be absolutely accurate? Designing
away the slight inaccuracy would be the nicest way of proceeding

 

[colin]

Well, there will be *some* more optimisation performed by the C#
compiler - but you'd presumably be using the same DirectX code either
way.

looking in the amd code analyst, it seems the xna.framework has a .jit
extension
like my application where as other modules dont ...

I have to say, I'm pretty much in the dark when it comes to DirectX -
it could be that there are oddities with what DirectX does when a
debugger is attached.


Hmm... that's not what I'd expect, to be honest. I'd expect it the
other way round, with higher precision results in the more optimised
code due to it not doing a conversion in between.

hmm Id rather not have to delve into assembly to see whats going on,
I dont even know how to as I just have the express edition.

That makes sense. Does it have to be absolutely accurate? Designing
away the slight inaccuracy would be the nicest way of proceeding

7 dec digits of accuracy is plenty enough to display pixels on the screem,
however the models are seperate and undergo numerous transformations,
when the transformed models are combined into one and hidden surfaces
removal is performed,
a problem arrises when the models are just touching,
with the limited resolution in the model data and the transform data and the
errors introduced
in calcualtions, models wich touch at a point or line or surface
are not going to be actually at exactly the same point,
so I allow for a MinDistance wich I set at about 0.1,
below wich I assume they are touching.

if the distance is close to 0.1 then theres a chance a slight difference in
math operation would alter the layout of the models wich is what im seeing.
although theres actually more visible errors than just this,
but it may all be a result of the same sort of thing.

whatever acuracy I use internally there will always be some chance of
something being being close to MinDistance,
due to the input data being non ideal, but this is something I need to be
able to let the user fix.

I would realy like to know whats actualy going on,
although if improving the maths makes it go away ....

Colin =^.^=