My project contained a folder that had like 100k files/folders inside. This was making indexing way too slow.
To fix this, exclude the folder from Project structure in PyCharm.
File - Settings - Project: yourprojectname - Project Structure - Right click on folder and press "Excluded"