Re: This calculation is just wrong / computer can't count! -------------------------------------------------------------------------------------------- Newsgroup: microsoft.public.vc.mfc Posted by: Geeky Badger 2007-10-08 17:04:46

GT,

You are just being silly. Think of the float type like you would a slide rule. It gives you the best answer it can with the precision available. It can't be more accurate than the number of bits it has to work with. It is up to you to pick a computational type on the computer with the correct degree of precision. It is the programmer's job to manage the rounding errors. That is you in this case.

The advantage of float is that it can give you an answer accurate to 6 and a half decimal digits over a huge scale. Whether the answer is 83333 with 38 zeros following it, or 38 zeros after the decimal point then 83333, it is as correct as it can be given the amount of storage which you invested in it.

The amount of storage matters when you start putting these things in a database.

If your goal is to accurately compute the answer for numbers in a known range such as adding up monetary amounts then floating point is a bad choice since it doesn't advertise this sort of accuracy. It only claims to deliver an approximate answer, but it can do so for very very small and very very large quantities.

-GB

On Mon, 8 Oct 2007 10:53:26 +0100, "GT" wrote: > >I expected the C++ basic data types to be able to handle simple maths >properly. Clearly it can only store digits to a certain number binary >places, therefore the trailing digit in a decimal conversion is unreliable >and 9/10 times mathematically *WRONG*. I expected there to be some logic >built in to the data type to take care of this. Would you buy a calculator >that gave wrong results? That is reality! >