View previous topic :: View next topic
|
Author |
Message |
nidhi.palsule
New User
Joined: 17 Feb 2009 Posts: 9 Location: Pune
|
|
|
|
I am converting a string '9.70000000' to Float Bin (53) using the builtin function FLOAT. My problem is that it gets converted to 9.699999809265137E+00.
Can someone please tell me why and provide me a solution to retain the exact value in float? |
|
Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007 Posts: 10872 Location: italy
|
|
|
|
by definition floating point values ( as a general case ) will never be exact values |
|
Back to top |
|
|
Robert Sample
Global Moderator
Joined: 06 Jun 2008 Posts: 8696 Location: Dubuque, Iowa, USA
|
|
|
|
Floating point values do not always exactly represent a mathematical value. Obviously, 9.7 is one of those values. The reasons have to do with the way the computer stores numbers and the fact that computers cannot store an infinite number of digits.
Unfortunately, there is no way to retain the exact value in floating point. This is one of the drawbacks to dealing with floating point. |
|
Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007 Posts: 10872 Location: italy
|
|
|
|
and to add to Robert considerations
there are base10 numbers which cannot be represented as base2 numbers
( fractional numbers of course ) |
|
Back to top |
|
|
nidhi.palsule
New User
Joined: 17 Feb 2009 Posts: 9 Location: Pune
|
|
|
|
But there are only a few cases where I have faced this problem, only one data record out of millions and that too only once or twice a year.
Also when I directly move a character string into a floating point variable, it retains the values, but when I use the function FLOAT to convert the value changes. |
|
Back to top |
|
|
Akatsukami
Global Moderator
Joined: 03 Oct 2009 Posts: 1788 Location: Bloomington, IL
|
|
|
|
Note that some, but not all, decimal numbers can not be accurately represented as floating point. We'd really need to see the usage of the FLOAT function, the source code, the compiler options, and the data (and a few benjamins -- I think that USD 125/hour is an average billing rate for us senior types) to determine exactly what is happening in each case. |
|
Back to top |
|
|
|