How to find the master URL for an existing spark cluster December 28, 2023 by Tarik I found that doing --master yarn-cluster works best. this makes sure that spark uses all the nodes of the hadoop cluster.