Use wildcard, e.g. replace 2008 with *:
df = sqlContext.read
.format("com.databricks.spark.csv")
.option("header", "true")
.load("../Downloads/*.csv") // <-- note the star (*)
Spark 2.0
// these lines are equivalent in Spark 2.0
spark.read.format("csv").option("header", "true").load("../Downloads/*.csv")
spark.read.option("header", "true").csv("../Downloads/*.csv")
Notes:
-
Replace
format("com.databricks.spark.csv")by usingformat("csv")orcsvmethod instead.com.databricks.spark.csvformat has been integrated to 2.0. -
Use
sparknotsqlContext