Segmentation fault when popping x86 stack to access function arg

This is a possible assembly code of int x = plus_10(40);

        push    40                      ; push argument
        call    plus_10                 ; call function
retadd: add     esp, 4                  ; clean up stack (dummy pop)
        ; result of the function call is in EAX, per the calling convention

        ; if compiled without optimization, the caller might just store it:
        mov     DWORD PTR [ebp-x], eax  ; store return value
                                        ; (in eax) in x

Now when you call plus_10, the address retadd is pushed on the stack by the call instruction. It’s effectively a push+jmp, and ret is effectively pop eip.

So your stack looks like this in the plus_10 function:

|  ...   |
+--------+
|   40   |  <- ESP+4 points here (the function argument)
+--------+
| retadd |  <- ESP points here
+--------+

ESP points to a memory location that contains the return address.

Now if you use pop edx the return address goes into edx and the stack looks like this:

|  ...   |
+--------+
|   40   |  <- ESP points here
+--------+

Now if you execute ret at this point, the program will actually jump to address 40 and most likely segfault or behave in some other unpredictable way.

The actual assembly code generated by the compiler may be different, but this illustrates the problem.


BTW, a more efficient way to write your function is this: it’s what most compilers would do with optimization enabled, for a non-inline version of this tiny function.

global plus_10
plus_10:
    mov   eax,  [esp+4]    ; retval = first arg
    add   eax,  10         ; retval += 10
    ret

This is smaller and slightly more efficient than

    mov   eax,  10
    add   eax,  [esp+4]        ; decode to a load + add.
    ret

Leave a Comment

tech