If you really do want to just delete the .svn directories, this could help:
gci c:\yourdirectory -include .svn -Recurse -Force |
Remove-Item -Recurse -Force
Edit:
Added -Force param to gci to list hidden directories and shortened the code.
Keith is right that it you need to avoid deleting files with .svn extension, you should filter the items using ?.