How to round off till most significant byte

star_dhruv2000 · Posted: Sat Jan 12, 2008 6:54 pm

Hi all,

I have following data definitions

Craq Giegerich · Posted: Sat Jan 12, 2008 10:17 pm

mmwife · Super Moderator Joined: 30 May 2003 Posts: 1592

Hi Star,

Craq Giegerich · Posted: Mon Jan 14, 2008 12:28 am

When rounding to 2 decimal places only the third decimal place is tested to see if the second decimal place needs to be changed. Find an elementary school math book and look it up.

Amit Banerjee · New User Joined: 12 Jan 2008 Posts: 14 Location: india

hi Dhruv,
the rule with 5 is that if the next digit is <5 then it's rounded to the same value but if it's greater than 5 it's rounded to the next higher value.
i.e if u round 0.225 to 2 places u'll get 0.22
but for 0.275 u'll get 0.28.

Craq Giegerich · Posted: Tue Jan 15, 2008 1:48 am

stodolas · Posted: Tue Jan 15, 2008 2:02 am

Rounding is done by first truncating one position past the place you want to round to then applying rounding rules.

.222222222222222225 rounded to 2 places is evaluated like this:
.222
with a final result of .22

.2225
.222
with a final result of .22

.225999999999999
.225
with a final result of .23

This is how it works when done by hand or a computer.

Amit Banerjee · New User Joined: 12 Jan 2008 Posts: 14 Location: india

in the 1st case since 2<5 it's rounded off to 2 only and the result is 0.22.
in the 2nd case 7>5 and thus gets rounded off to 8 resulting to 0.28.
That's what's happenning in dhruv's case also
initial 0.1244444444444447
since last digit is 7,4 is rounded off to 5
thus 0.124444444444445
now since last digit is 5 and 4<5,it gets rounded to 4 giving
0.12444444444444
now it remains 4 till the end and the final number is 0.12

stodolas · Posted: Tue Jan 15, 2008 2:14 am

Amit - you are wrong. Whether or not a position in the number is rounded has nothing to do with the value of that position, only the digit to the right positionally has an affect on rounding.

The OP is wants to see "chain rounding" which is not how rounding works. Only the digit to the direct right of the position to be rounded affects its value.

"Round 436.449 to the nearest whole number.

Chain rounding would round 436.449 to 436.45 and then to 436.5 and then to 437. That is not correct. Instead, round only based on the number closest to the number you want to round to. In this case, the first 4 after the decimal point is the key; therefore, the answer is 436."

See the rules of rounding here: www.dtc.dla.mil/qa/RR.htm

stodolas · Posted: Tue Jan 15, 2008 2:25 am

From the COBOL manual - that the OP should have consulted.
When the size of the fractional result exceeds the number of places provided for its storage, truncation occurs unless ROUNDED is specified. When ROUNDED is specified, the least significant digit of the resultant identifier is increased by 1 whenever the most significant digit of the excess is greater than or equal to 5.

The most significant digit of the excess in the OP's example is 4 and therefor does not cause the value to round to .13. If he wants to do chain rounding (which is completely wrong mathematically) then he needs to define vars for each size all the way down to the result he wants

Phrzby Phil · Posted: Tue Jan 15, 2008 6:52 am

Wow. This stuff should be elementary.

You can string as many digits as you want to the right in you examples, but as Steve and others have said, the determining digit is the one immediately to the right of the position being rounded to.

My credentials: adjunct mathematics instructor at a community college, where I teach a remedial math class.

stodolas · Posted: Tue Jan 15, 2008 7:17 am

1 Advanced Math class shy of a math minor. I was away from the calculus for 4 years and didn't use it. All the classes I needed required working knowledge of calc

Now I just read math books for entertainment.

Phrzby Phil · Posted: Tue Jan 15, 2008 9:59 pm

Steve -

I heartily recommend the classic 4-vol set The World of Mathematics, ed. by James R. Newman. 2500 pages from ancient to modern (as of 1956, but who's counting (pun intended)?). Original sources.