Rounding whole numbers help.

ID:2385959
 
Jul 20 2018, 10:02 pm
(See the best response by Kaiochao.)
Code:
var/oldWidth = 28
var/newWidth = oldWidth * 1.5

var/XscaleFactor = (oldWidth - 1) / (newWidth - 1)

for(var/count_x = 1; count_x <= oldWidth * 1.5; count_x++)
    var/xVal = (count_x - 1) * XscaleFactor + 1

    var/roundedX = round(xVal)

    usr << "[count_x] gives an xVal of [xVal] with a roundedX of [roundedX]" //DEBUG TO CHECK THE NUMBERS

    if(xVal != roundedX)
        alert("Not a whole number")
        usr << "Not a Whole Number"
    else
        alert("A whole number")
        usr << "Whole Number"


Problem description:
I decided to post here before I post a bug report as my code may be wrong.

The above code should loop 42 (28 * 1.5) times and call alert("A whole number") twice. Once when count_x is 1 and when count_x is 42

This is because at count_x = 1, xVal = 1 and at count_x = 42, xVal = 28. Which is true, but the if statement stops working for count_x = 42 for some reason. I put in a debug line to check the values and the roundedX and xVal are both exactly 28 when count_x is 42, so it should call the whole number but it doesn't...

Any input?

EDIT: It's basically saying 28 != 28 and I have no idea why...
Jul 20 2018, 10:10 pm
After doing a couple of tests, it turns out xVal is coming up as 28, but is not actually the whole number 28.

Maybe an internal rounding error?

if(xVal == 28)
   alert("YAY")
else
   alert("NAY")


I get Nay here.

But if I compare with the whole number.

if(roundedX == 28)
   alert("YAY")
else
   alert("NAY")


I get YAY here.

Is this a bug?
Jul 20 2018, 11:37 pm (Edited on Jul 20 2018, 11:49 pm)
Best response
It's probably 28.000001 or 27.9999999 or something. You can check by outputting num2text(xVal, 20) instead of just xVal.

Yes, floating point numbers are inherently limited in precision. It's not really a bug... it can't really be fixed, just worked around.

If you inline the XscaleFactor like so:
for(var/count_x = 1 to newWidth) // <-- also you can just do this
    var/xVal = (count_x - 1) * (oldWidth - 1) / (newWidth - 1) + 1

Then the multiplication can come before the division, so you're less likely to result in a non-whole number at any point. For example, by stepping through the calculation and following the order of operations:

count_x = 1, old_width = 28, new_width = 42:
(count_x - 1) * (oldWidth - 1) / (newWidth - 1) + 1 = 0 * 27 / 42 + 1 = 0 / 42 + 1 = 0 + 1 = 1

count_x = 42:
41 * 27 / 41 + 1 = 1107 / 41 + 1 = 27 + 1 = 28
There's no non-whole number involved.

Previously, you had an XscaleFactor equal to 27/41, which is approximately 0.658536585.
When you put 41 * 0.658536585 into Google, you actually get 26.999999985. That's a problem.
Jul 20 2018, 11:49 pm
Ah I see, that makes sense and works nicely.

Also, thanks for the tip in the loop, I didn't know we could do that!

Thanks!
Jul 21 2018, 12:15 pm
I have one question about this. Would it be a problem to have used round()? I use it all the time. Does using in a loop cause lag, if its a big loop?
Jul 21 2018, 8:48 pm
I tried it again rouding to 3 decimal places ( round(value, 0.001) ) and it works, But Kaiochao's method is better as there was no need to calculate the scale factor seperately in my case.