A little late but you could also use build args and change your code to:
FROM python:3.6.5-stretch
[ ... ]
ARG MY_SERVICE_PORT=8080
ARG MY_SERVICE_PORT_RPC=50051
# 8080 and 50051 would be the defaults
[ ... ]
# Still functions like environment variables :)
EXPOSE ${MY_SERVICE_PORT}
EXPOSE ${MY_SERVICE_PORT_RPC}
Then you can build with docker build --build-arg MY_SERVICE_PORT=80 -t image_tag
before you run.
This way you could have your containerized application and your container running with the same ports without getting too complex.