Read files sent with spark-submit by the driver

Yes, you can access files uploaded via the --files argument.

This is how I’m able to access files passed in via --files:

./bin/spark-submit \
--class com.MyClass \
--master yarn-cluster \
--files /path/to/some/file.ext \
--jars lib/datanucleus-api-jdo-3.2.6.jar,lib/datanucleus-rdbms-3.2.9.jar,lib/datanucleus-core-3.2.10.jar \
/path/to/app.jar file.ext

and in my Spark code:

val filename = args(0)
val linecount = Source.fromFile(filename).getLines.size

I do believe these files are downloaded onto the workers in the same directory as the jar is placed, which is why simply passing the filename and not the absolute path to Source.fromFile works.