Spark / Scala: forward fill with last observation
Initial answer (a single time series assumption): First of all try avoid window functions if you cannot provide PARTITION BY clause. It moves data to a single partition so most of the time it is simply not feasible. What you can do is to fill gaps on RDD using mapPartitionsWithIndex. Since you didn’t provide an … Read more