docker-compose down with a non-default yml file name

You should run the docker-compose down command with the same flags that when you started the containers.

docker-compose -f docker-compose-test.yml down