You probably didn’t fully deactivate the Conda environment – remember, the command you need to use with Conda is conda deactivate (for older versions, use source deactivate). So it may be wise to start a new shell and activate the environment in that before you try. Then deactivate it.
You can use the command
conda env remove -n ENV_NAME
to remove the environment with that name. (--name is equivalent to -n)
Note that you can also place environments anywhere you want using -p /path/to/env instead of -n ENV_NAME when both creating and deleting environments, if you choose. They don’t have to live in your conda installation.
UPDATE, 30 Jan 2019: From Conda 4.6 onwards the conda activate command becomes the new official way to activate an environment across all platforms. The changes are described in this Anaconda blog post