[*] RDDs extend the Serialisable interface, so this is not what’s causing your task to fail. Now this doesn’t mean that you can serialise an RDD with Spark and avoid NotSerializableException Spark is a distributed computing engine and its main abstraction is a resilient distributed dataset (RDD), which can be viewed as a distributed collection. … Read more
The example you have given only uses call-by-value, so I will give a new, simpler, example that shows the difference. First, let’s assume we have a function with a side-effect. This function prints something out and then returns an Int. def something() = { println(“calling something”) 1 // return value } Now we are going … Read more
For most cases you do not need an external parser. Scala’s pattern matching allows consuming args in a functional style. For example: object MmlAlnApp { val usage = “”” Usage: mmlaln [–min-size num] [–max-size num] filename “”” def main(args: Array[String]) { if (args.length == 0) println(usage) val arglist = args.toList type OptionMap = Map[Symbol, Any] … Read more
The difference between them is, that a val is executed when it is defined whereas a lazy val is executed when it is accessed the first time. scala> val x = { println(“x”); 15 } x x: Int = 15 scala> lazy val y = { println(“y”); 13 } y: Int = <lazy> scala> x … Read more
This is my working solution using old-style macro annotations. import scala.language.experimental.macros import scala.reflect.macros.blackbox.Context import scala.annotation.StaticAnnotation class fieldable extends StaticAnnotation { def macroTransform(annottees: Any*): Any = macro fieldableMacro.impl } object fieldableMacro { def impl(c: Context)(annottees: c.Expr[Any]*): c.Tree = { import c.universe._ annottees.map(_.tree) match { case (param @ q”case class $className(..$fields)”) :: Nil => { val implicits … Read more
I thought this was asked already, but, if so, the question isn’t apparent in the “related” bar. So, here it is: What is a View Bound? A view bound was a mechanism introduced in Scala to enable the use of some type A as if it were some type B. The typical syntax is this: … Read more
You have three (or so) options to break out of loops. Suppose you want to sum numbers until the total is greater than 1000. You try var sum = 0 for (i <- 0 to 1000) sum += i except you want to stop when (sum > 1000). What to do? There are several options. … Read more
Let me make up for starting some of this confusion by pitching in with some disambiguation. I like to use the analogy to the value level to explain this, as people tend to be more familiar with it. A type constructor is a type that you can apply to type arguments to “construct” a type. … Read more
I must say that the example copied out of the Scala documentation by skaffman above is of limited utility in practice (you might as well use case objects). In order to get something most closely resembling a Java Enum (i.e. with sensible toString and valueOf methods — perhaps you are persisting the enum values to … Read more
As so many others have said, the object assigned to a val cannot be replaced, and the object assigned to a var can. However, said object can have its internal state modified. For example: class A(n: Int) { var value = n } class B(n: Int) { val value = new A(n) } object Test … Read more