If you are you using WSL2, indeed it could be related to pip trying to connect to an XServer. If so, clearing the DISPLAY environment variable first before running it may help:
export DISPLAY=
pip install <packagename>
(Or, as a one-liner: DISPLAY= pip install <packagename>)