Understand Spark: Cluster Manager, Master and Driver nodes
1. The Cluster Manager is a long-running service, on which node it is running? Cluster Manager is Master process in Spark standalone mode. It can be started anywhere by doing ./sbin/start-master.sh, in YARN it would be Resource Manager. 2. Is it possible that the Master and the Driver nodes will be the same machine? I … Read more