It is not possible. As for now (Spark 1.6.0 / 2.2.0 SNAPSHOT) Spark DataFrameWriter supports only four writing modes: SaveMode.Overwrite: overwrite the existing data. SaveMode.Append: append the data. SaveMode.Ignore: ignore the operation (i.e. no-op). SaveMode.ErrorIfExists: default option, throw an exception at runtime. You can insert manually for example using mapPartitions (since you want an UPSERT … Read more
This person was having similar issue: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-DataFrame-with-MySQL-td22178.html Have you updated your connector drivers to the most recent version? Also did you specify the driver class when you called load()? Map<String, String> options = new HashMap<String, String>(); options.put(“url”, “jdbc:mysql://localhost:3306/video_rcmd?user=root&password=123456”); options.put(“dbtable”, “video”); options.put(“driver”, “com.mysql.cj.jdbc.Driver”); //here DataFrame jdbcDF = sqlContext.load(“jdbc”, options); In spark/conf/spark-defaults.conf, you can also set spark.driver.extraClassPath … Read more
This one is pretty commonly used, works with any SQL database, and is pretty well supported: https://github.com/julianhyde/sqlline
Ok, I did a little research and found this in the changelog (http://www.h2database.com/html/changelog.html): Implicit relative paths are disabled (system property “h2.implicitRelativePath”), so that the database URL jdbc:h2:test now needs to be written as jdbc:h2:./test. In H2 starting from version 1.4.177 Beta, implicit relative paths are not allowed anymore. Therefore, in your case the url should … Read more
You’ve done good research. I’d say, go ahead with BoneCP. Some years ago I would have suggested c3p0, but it has currently some open and unresolved issues as well. Too bad, actually. The BoneCP author is by the way also active here at SO.