This command almost gives you what you want, and it even works with a remote server. The only caveat is that it generates a TSV file (fields are separated by a tab).
mysql mydb -e "select * from mytable" -B > mytable.tsv
But you could convert it to CSV using sed, as suggested in this answer:
mysql mydb -e "select * from mytable" -B | sed "s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g" > mytable.csv