Double cast to unsigned int on Win32 is truncating to 2,147,483,648

A compiler bug…

From assembly provided by @anastaciu, the direct cast code calls __ftol2_sse, which seems to convert the number to a signed long. The routine name is ftol2_sse because this is an sse-enabled machine – but the float is in a x87 floating point register.

; Line 17
    call    _getDouble
    call    __ftol2_sse
    push    eax
    push    OFFSET ??_C@_0BH@GDLBDFEH@Direct?5cast?5value?3?5?$CFu?6@
    call    _printf
    add esp, 8

The indirect cast on the other hand does

; Line 18
    call    _getDouble
    fstp    QWORD PTR _d$[ebp]
; Line 19
    movsd   xmm0, QWORD PTR _d$[ebp]
    call    __dtoui3
    push    eax
    push    OFFSET ??_C@_0BJ@HCKMOBHF@Indirect?5cast?5value?3?5?$CFu?6@
    call    _printf
    add esp, 8

which pops and stores the double value to the local variable, then loads it into a SSE register and calls __dtoui3 which is a double to unsigned int conversion routine…

The behaviour of the direct cast does not conform to C89; nor does it conform to any later revision – even C89 explicitly says that:

The remaindering operation done when a value of integral type is converted to unsigned type need not be done when a value of floating type is converted to unsigned type. Thus the range of portable values is [0, Utype_MAX + 1).


I believe the problem might be a continuation of this from 2005 – there used to be a conversion function called __ftol2 which probably would have worked for this code, i.e. it would have converted the value to a signed number -2147483647, which would have produced the correct result when interpreted an unsigned number.

Unfortunately __ftol2_sse is not a drop-in replacement for __ftol2, as it would – instead of just taking the least-significant value bits as-is – signal the out-of-range error by returning LONG_MIN / 0x80000000, which, interpreted as unsigned long here is not at all what was expected. The behaviour of __ftol2_sse would be valid for signed long, as conversion of a double a value > LONG_MAX to signed long would have undefined behaviour.

Leave a Comment

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)