Based on your example, it looks like you are trying to do something akin to always deleting a temporary file, regardless of how a script exits. In Bash to do this try the trap builtin command to trap the EXIT signal.
#!/bin/bash
trap 'rm tmp' EXIT
if executeCommandWhichCanFail; then
mv output
else
mv log
exit 1 #Exit with failure
fi
exit 0 #Exit with success
The rm tmp statement in the trap is always executed when the script exits, so the file “tmp” will always tried to be deleted.
Installed traps can also be reset; a call to trap with only a signal name will reset the signal handler.
trap EXIT
For more details, see the bash manual page: man bash