Why is -march=native used so rarely?

Conservative

If you take a closer look at the defaults of gcc, the oldest compiler in your list, you’ll realize that they are very conservative:

• By default, on x86, only SSE 2 is activated; not even SSE 4.
• The set of flags in -Wall and -Wextra has not changed for years; there are new useful warnings, they are NOT added to -Wall or -Wextra.

Why? Because it would break things!

There are entire development chains relying on those convenience defaults, and any alteration brings the risk of either breaking them, or of producing binaries that will not run on the targets.

The more users, the greater the threat, so developers of gcc are very, very conservative to avoid world-wide breakage. And developers of the next batch of compilers follow in the footsteps of their elders: it’s proven to work.

Note: rustc will default to static linking, and boasts that you can just copy the binary and drop it on another machine; obviously -march=native would be an impediment there.

Masses Friendly

And in the truth is, it probably doesn’t matter. You actually recognized it yourself:

In my own experience, this flag can provide massive speedups for numerically-intensive code

Most code is full of virtual calls and branches (typically OO code) and not at all numerically-intensive. Thus, for the majority of the code, SSE 2 is often sufficient.

The few codebases for which performance really matters will require significant time invested in performance tuning anyway, both at code and compiler level. And if vectorization matters, it won’t be left at the whim of the compiler: developers will use the built-in intrinsics and write the vectorized code themselves, as it’s cheaper than putting up a monitoring tool to ensure that auto-vectorization did happen.

Also, even for numerically intensive code, the host machine and the target machine might differ slightly. Compilation benefits from lots of core, even at a lower frequency, while execution benefits from a high frequency and possibly less cores unless the work is easily parallelizable.

Conclusion

Not activating -march=native by default makes it easier for users to get started; since even performance seekers may not care for it much, this means there’s more to lose than gain.

In an alternative history where the default had been -march=native from the beginning; users would be used to specify the target architecture, and we would not be having this discussion.