As of spark 1.2.0 you can set memory
and cores
by giving following arguments to spark-shell.
spark-shell --driver-memory 10G --executor-memory 15G --executor-cores 8
to see other options you can give following commands to spark shell
spark-shell --help