Why is address 0x400000 chosen as a start of text segment in x86_64 ABI?

Bottom line: some technical limitations that amd64 has in using large addresses suggest dedicating the lower 2GiB of address space to code and data for efficiency. Thus the stack has been relocated out of this range.

In i386 ABI¹

• stack is located before the code, growing from just under 0x8048000 downwards. Which provides “a little over 128 MB
  for the stack and about 2 GB for text and data” (p. 3-22).
• Dynamic segments start at 0x80000000 (2GiB),
• and the kernel occupies the “reserved area” at the top which the spec allows to be up to 1GiB, starting at at least 0xC0000000 (p. 3-21) (which is what it typically does).
• The main program is not required to be position-independent.
• An implementation is not required to catch null pointer access (p. 3-21) but it’s reasonable to expect that some of the stack space above 128MiB (which is 288KiB) will be reserved for that purpose.

amd64 (whose ABI is formulated as an amendment to the i386 one (p. 9)) has a vastly bigger (48-bit) address space but most instructions only accept 32-bit immediate operands (which include direct addresses and offsets in jump instructions), requiring more work and less efficient code (especially when taking instruction interdependency into consideration) to handle larger values. Measures to work around these limitations are summarized by the authors by introducing a few “code models” they recommend to use to “allow the compiler to generate better code”. (p. 33)

• Specifically, the first of them, “Small code model”, suggests using addresses “in the range from 0 to 2³¹-2²⁴-1 or from 0x00000000 to 0x7effffff“ which allows some very efficient relative references and array iteration. This is 1.98GiB which is more than enough for many programs.
• “Medium code model” is based on the previous one, splitting the data into a “fast” part under the above boundary and the “slower” remaining part which requires a special instruction to access. While code remains under the boundary.
• And only the “large” model makes no assumptions about sizes, requiring the compiler “to use the movabs instruction, as in the medium
  code model, even for dealing with addresses inside the text section. Additionally, indirect branches are needed when branching to addresses whose
  offset from the current instruction pointer is unknown.” They go on to suggest splitting the code base into multiple shared libraries since these measures do not apply for relative references with offsets that are known to be within bounds (as outlined in “Small position independent code model”).

Thus the stack was moved to under the shared library space (0x80000000000, 128GiB) because its addresses are never immediate operands, always referenced either indirectly or with lea/mov from another reference, thus only relative offset limitations apply.

The above explains why the loading address was moved to a lower address. Now, why was it moved to exactly 0x400000 (4MiB)? Here, I came empty so, summarizing what I’ve read in the ABI specs, I can only guess that it felt “just right”:

• It’s large enough to catch any likely incorrect structure offset, allowing for larger data units that amd64 operates on, yet small enough to not waste much of the valuable starting 2GiB of address space.
• It’s equal to the largest practical page size to date and is a multiple of all other virtual memory unit sizes one can think of.

¹_{Note that actual x32 Linuxes have been deviating from this layout more and more as time goes. But we’re talking about the ABI spec here since the amd64 one is formally based on it rather than any derived layout (see its paragraph for citation).}