JasmineCorp Directory   
  
Usenet News Group Archives!!!

Usenet Groups:

 

 

 

 

 

Re: This calculation is just wrong / computer can't count!
--------------------------------------------------------------------------------------------
Newsgroup: microsoft.public.vc.mfc
Posted by: Norbert Unterberg
2007-10-08 17:01:40

GT schrieb:

> I am not clinging to anything, I'm simply doing some basic maths using a
> basic data type designed to handle numbers with digits after the decimal
> point and the result being handed back is mathematically very precise (18
> digits?), but inaccurate (wrong). If it stopped retrieval at the 17th digit,
> so that the final digit is valid, then the answer would be precise to 16 or
> 17 digits and accurate (mathematically correct). This error occurs many
> steps before I get a final number for display.

Thwe "problem" is that double is *not* a datatype designed to handle digits
after the decimal point. double does not know about decimal digits. double has a
52 *bit* mantissa (that means binary digits). After doing the calculations, all
52 bits are as correct as they can be (the "correct" result is beeing rounded to
52 binary digits). With 52 binary digits, you can represent numbers up to
log10(2^52) digits on printout, which is around 15.6. But these digits only
matter when representing numbers to the user, not when doing calculations.
There is no "retrieval" of a value that can be stopped at some decimal digit.
All binary digits are correct.

doubles can not represent numbers "mathematically correct". That is the reason
why 25 - (25/30) * 30 != 0. On each operation there are rounding errors. Your
algrithm must be prepared for these errors.

You say your algorithm does not need more that 5 to 6 digits precision (or was
it accuracy?). That means that you would be perfectly fine with using doubles.
Only when you do the final printout, you would use something like
printf("%.6f", result);

Or, if you are concerned about results like "-0.0", you could add a rounding and
result cleanup stage at the end of your algorithm.

Maybe it is easier to look at all this from a different point of view. How
precise needs your result to be, what is the largest error you can afford. The
error you get (and what you see in the 17th decimal digit) is DBL_EPSILON (from
float.h). It is about 2.2e-16. It seems that you would be fine with 1e-6 or
1e-7, so 2e-16 should be just fine.

Norbert

 

 

 

More >> 

Domain Registration:
.com .org .net
.info .biz .us