I had exactly the same issue with my brand new Mac Mini. After spending an afternoon reading this issue on github, I finally succeeded. However, it only seems to work with v1.20.0rc2 or above…
Long story short, these were the steps I’ve taken to get it working:
pip3 install Cython
git clone https://github.com/numpy/numpy.git
cd numpy
pip3 install . --no-binary :all: --no-use-pep517
After that, I could cd into my virtualenv and import numpy:
>>> import numpy as np
>>> np.__version__
'1.21.0.dev0+402.gc4ae3ce64'
>>>
Next up: pandas.
UPDATE:
After some more searching, I found this interesting article, which states that you can easily start Terminal in Rosatta mode. That also solved the issues.
However, I am working with a headless Mac Mini, so I still have to find out whether that solution also is feasable in that particular setup.
UPDATE 2
I now have pandas working as well, thanks to this answer on SO.