There are a few things going on. First, Scala allows dots and parens to be omitted from many method calls, so 20 seconds is equivalent to 20.seconds()*. Second, an “implicit conversion” is applied. Since 20 is an Int and Int has no seconds method, the compiler searches for an implicit conversion that takes an Int … Read more
Just double it scala> val name = “foo” name: String = foo scala> s”my.package.$name$$” res0: String = my.package.foo$
It’s worth explicitly mentioning that Auxiliary Constructors in Scala must either call the primary constructor (as in landon9720’s) answer, or another auxiliary constructor from the same class, as their first action. They cannot simply call the superclass’s constructor explicitly or implicitly as they can in Java. This ensures that the primary constructor is the sole … Read more
It enables one to bind a matched pattern to a variable. Consider the following, for instance: val o: Option[Int] = Some(2) You can easily extract the content: o match { case Some(x) => println(x) case None => } But what if you wanted not the content of Some, but the option itself? That would be … Read more
Spark >= 2.3.0 SPARK-22614 exposes range partitioning. val partitionedByRange = df.repartitionByRange(42, $”k”) partitionedByRange.explain // == Parsed Logical Plan == // ‘RepartitionByExpression [‘k ASC NULLS FIRST], 42 // +- AnalysisBarrier Project [_1#2 AS k#5, _2#3 AS v#6] // // == Analyzed Logical Plan == // k: string, v: int // RepartitionByExpression [k#5 ASC NULLS FIRST], 42 … Read more
This code works perfectly from Spark 2.x with Scala 2.11 Import necessary classes import org.apache.spark.sql.{Row, SparkSession} import org.apache.spark.sql.types.{DoubleType, StringType, StructField, StructType} Create SparkSession Object, and Here it’s spark val spark: SparkSession = SparkSession.builder.master(“local”).getOrCreate val sc = spark.sparkContext // Just used to create test RDDs Let’s an RDD to make it DataFrame val rdd = sc.parallelize( … Read more
All three forms are valid, and have the effect that B is assumed as the type of this in class A. The first two variants trait A { self: B => … } trait A { foo: B => … } introduce self (respectively, foo) as an alias for this in trait A. This is … Read more
To explain it, we first have to explain nested classes in Scala. Consider this simple example: class A { class B def f(b: B) = println(“Got my B!”) } Now let’s try something with it: scala> val a1 = new A a1: A = A@2fa8ecf4 scala> val a2 = new A a2: A = A@4bed4c8 … Read more
There are three basic ways how to create a project – modern versions of IntelliJ can import sbt project out of the box, otherwise you can either use sbt plugin to generate IntelliJ project, or use IntelliJ Scala plugin to create sbt project. Basic features work out of the box using both solutions, some complex … Read more
Okay, I had to add one more. Every Regex object in Scala has an extractor (see answer from oxbox_lakes above) that gives you access to the match groups. So you can do something like: // Regex to split a date in the format Y/M/D. val regex = “(\\d+)/(\\d+)/(\\d+)”.r val regex(year, month, day) = “2010/1/13” The … Read more