You could just use Read The Docs to host your documentation for you. They automatically handle multiple versions with a dropdown:
https://docs.readthedocs.io/en/latest/index.html
If you’d like to host your own documentation on GitHub Pages or some other web server, I made a Sphinx extension that does what you’re looking for:
https://github.com/sphinx-contrib/sphinxcontrib-versioning