libc: fix 32-bit arch vlong->double conversion (thans cosa)

Cosa wrote:

While trying to run kvik's lu9 on ARM, I found that when converting
LLONG_MIN to a double on 32bit systems, the result is wrong (positive
instead of negative).

Given the following test program:

void main() {
     vlong min = LLONG_MIN;
     double dmin = min;
     print("minint %lld\n", min);
     print("minint as double %f\n", dmin);
     if (dmin > 0.0) {
         exits("int min as double turned positive");
     }
     exits(0);
}

The output on x86_64 will be:

minint -9223372036854775808
minint as double -9223372036854776400.000000

But on arm or 386 (and I expect also spim, 68000, mips, 68020, sparc,
power, since they all use the same _v2d):

minint -9223372036854775808
minint as double 9223372036854776400.000000

And the value turned positive in the conversion.

The function used for the cast to double is (in /sys/src/libc/arm/vlrt.c):

double
_v2d(Vlong x)
{
     if(x.hi & SIGN(32)) {
         if(x.lo) {
             x.lo = -x.lo;
             x.hi = ~x.hi;
         } else
             x.hi = -x.hi;
         return -((long)x.hi*4294967296. + x.lo);
     }
     return (long)x.hi*4294967296. + x.lo;
}

If I understand correctly, the issue is that where it tries to flip the
sign for x.hi (x.hi = -x.hi), 0x80000000 has no positive, thus stays the
same (it stays negative). Then when we get to the negative return, we
get a positive out.

What came to my mind then, is that in the case that there is no x.lo, we
can keep the x.hi sign and cast directly, thus:

double
_v2d(Vlong x)
{
     if(!x.lo) {
         return (long)x.hi*4294967296.;
     }
     if(x.hi & SIGN(32)) {
         x.lo = -x.lo;
         x.hi = ~x.hi;
         return -((long)x.hi*4294967296. + x.lo);
     }
     return (long)x.hi*4294967296. + x.lo;
}

This looks correct to me, but I don't trust myself to not make mistakes
in such critical code, so I would like some feedback on the change.

Happy new year in advance,

cosa

1 files changed, 5 insertions, 5 deletions

diff --git a/sys/src/libc/386/vlrt.c b/sys/src/libc/386/vlrt.c
index 132a04f99..b27705baf 100644
--- a/sys/src/libc/386/vlrt.c
+++ b/sys/src/libc/386/vlrt.c
@@ -95,12 +95,12 @@ _f2v(Vlong *y, float f)
 double
 _v2d(Vlong x)
 {
+	if(!x.lo) {
+		return (long)x.hi*4294967296.;
+	}
 	if(x.hi & SIGN(32)) {
-		if(x.lo) {
-			x.lo = -x.lo;
-			x.hi = ~x.hi;
-		} else
-			x.hi = -x.hi;
+		x.lo = -x.lo;
+		x.hi = ~x.hi;
 		return -((long)x.hi*4294967296. + x.lo);
 	}
 	return (long)x.hi*4294967296. + x.lo;