|
View previous topic :: View next topic
|
| Author |
Message |
senthamizh
New User
Joined: 27 Apr 2009
Posts: 23
Location: mumbai
|
|
|
|
Hi,
in my program i am fetching the rate from a table.When i check tha table it is holding value of 17.38.but when i put display for host variable it is showing 17.37.This column is nullable column.
my query is
select ifnull(RATE,1 )
into :ratef
from table1.
RATE is defined as FLOAT.
i am not sure how 17.37 is retived. I have defined Host variable as RATE usasge comp2.
Can some one help me on this.How the table value is subracted by .01. |
|
| Back to top |
|
 |
Robert Sample
Global Moderator
Joined: 06 Jun 2008
Posts: 8697
Location: Dubuque, Iowa, USA
|
|
|
|
The value 17.38, expressed as a D-type floating point value in Assembler, is X'42116147AE147AE1' so it is obviously not a precise value in hexadecimal. The most likely explanation for what you are seeing is that COBOL and DB2 handle floating point values slightly differently, and you have found one of those values where the difference is apparent.
Unfortunately, this is one of the drawbacks to using floating point values, and you need to find out how your site handles such discrepancies. They can happen at any time for any value and there's not a lot that can be done about the difference, other than change the data base to not use floating point values. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| You'd better show your code. If you DISPLAY a COMP-2 you're not going to get the output you have shown. |
|
| Back to top |
|
|
Robert Sample
Global Moderator
Joined: 06 Jun 2008
Posts: 8697
Location: Dubuque, Iowa, USA
|
|
|
|
To add to Bill's comment, when I create a COMP-2 variable and move 17.38 to it for DISPLAY, I get
| Code: |
| .17379999999999999E 02 |
If this is why you think the value is 17.37, you need to move over to the Beginner's and Students forum instead of this one. |
|
| Back to top |
|
|
|
|
