After a brute force search, Jupyter stores kernel info for OS X in /Users/${USER}/Library/Jupyter/kernels. This list got copied across from my .ipython/kernels list hence renaming it made no difference.
Removing /Users/${USER}/Library/Jupyter/kernels fixes the issue.
This is the reference I was looking for: http://jupyter-client.readthedocs.org/en/latest/kernels.html#kernelspecs