OK I found it, needed to add --recursive when cloning the repo.
So the clone command ends up as:
git clone https://github.com/aikiframework/json.git --recursive
Note that if you forgot the --recursive flag you can do (thanks to @Amber):
git submodule update --init