When you do helm delete $RELEASE_NAME it deletes all resources but keeps the record with $RELEASE_NAME in case you want to rollback.
You can see removed releases via helm ls -a.
Whereas helm delete --purge $RELEASE_NAME removes records and make that name free to be reused for another installation.