Use the -set and formatting options.
convert *.psd -set filename:base "%[basename]" "%[filename:base].png"
See “Long Form Attribute Percent Escapes” and “Filename Percent Escapes” docs.
Update
The mogrify utility that ships with imagemagick can also be used.
mogrify -format png *.psd
Note: Be careful with mogrify as the docs state…
This tool is similiar to convert except that the original image file is overwritten (unless you change the file suffix with the -format option) with any changes you request.