Manual Assembly vs GCC

For those who wonder what the generated code comes from, first note that when GCC compile myAbs with stack protection it transform it into this form

long myAbs(long j) {
    uintptr_t canary = __stack_chk_guard;

    register long result = j < 0 ? -j : j;

    if ( (canary = canary ^ __stack_chk_guard) != 0 )
        __stack_chk_fail();
}

The code to simply perform j < 0 ? -j : j; is

movq    %rdi, %rdx     ;RDX = j
movq    %rdi, %rax     ;RAX = j
sarq    $63, %rdx      ;RDX = 0 if j >=0, 0fff...ffh if j < 0
xorq    %rdx, %rax     ;Note: x xor 0ff...ffh = Not X, x xor 0 = x
                       ;RAX = j if j >=0, ~j if j < 0
subq    %rdx, %rax     ;Note: 0fff...ffh = -1
                       ;RAX = j+0 = j if j >= 0, ~j+1 = -j if j < 0
                       ;~j+1 = -j in two complement

Analyzing the generated code we get

    pushq   %rbp
    movq    %rsp, %rbp       ;Standard prologue

    subq    $4144, %rsp      ;Allocate slight more than 4 KiB     
    orq     $0, (%rsp)       ;Perform a useless RW operation to test if there is enough stack space for __stack_chk_fail

    addq    $4128, %rsp      ;This leave 16 byte allocated for local vars

    movq    %rdi, %rdx       ;See above
    sarq    $63, %rdx        ;See above

    movq    %fs:40, %rax     ;Get the canary
    movq    %rax, -8(%rbp)   ;Save it as a local var
    xorl    %eax, %eax       ;Clear it

    movq    %rdi, %rax       ;See above
    xorq    %rdx, %rax       ;See above
    subq    %rdx, %rax       ;See above

    movq    -8(%rbp), %rcx   ;RCX = Canary
    xorq    %fs:40, %rcx     ;Check if equal to the original value
    jne     .L5              ;If not fail

    leave
    ret
.L5:
    call    __stack_chk_fail@PLT  ;__stack_chk_fail is noreturn

So all the extra instructions are for implementing the Stack Smashing Protector.

Thanks to FUZxxl for pointing out the use of the first instructions after the prologue.

Leave a Comment

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)