This may get closed as opinion based, but for what it’s worth my opinion is that docker should be involved in networking up to the transport layer (TCP/UDP) but no higher (HTTP). Thus I would say no you should not install nginx as a reverse proxy directly on your docker host directly and yes you should install nginx within your container(s) if you want the features nginx provides.
You could decide you want an nginx container in front of each of your applications (this seems most in harmony with the overall idea of containerization and separation and not sharing things), or one nginx container to reverse proxy several apps hosted in containers on the same host (simpler overall, but more tightly coupled and perhaps less harmonious with docker’s philosophy). You could also take the phusion style and put both your application and it’s nginx in the same container, which provides a nice coherent unit, but otherwise is against the common case of 1 process tree per container.