Unpivot in Spark SQL / PySpark

You can use the built in stack function, for example in Scala: scala> val df = Seq((“G”,Some(4),2,None),(“H”,None,4,Some(5))).toDF(“A”,”X”,”Y”, “Z”) df: org.apache.spark.sql.DataFrame = [A: string, X: int … 2 more fields] scala> df.show +—+—-+—+—-+ | A| X| Y| Z| +—+—-+—+—-+ | G| 4| 2|null| | H|null| 4| 5| +—+—-+—+—-+ scala> df.select($”A”, expr(“stack(3, ‘X’, X, ‘Y’, Y, ‘Z’, … Read more

SQL Server – Include NULL using UNPIVOT

To preserve NULLs, use CROSS JOIN … CASE: select a.ID, b.column_name , column_value = case b.column_name when ‘col1’ then a.col1 when ‘col2’ then a.col2 when ‘col3’ then a.col3 when ‘col4’ then a.col4 end from ( select ID, col1, col2, col3, col4 from table1 ) a cross join ( select ‘col1’ union all select ‘col2’ union … Read more

How do you create a “reverse pivot” in Google Sheets?

I wrote a simple general custom function, which is 100% reusable you can unpivot / reverse pivot a table of any size. In your case you could use it like this: =unpivot(A1:D4,1,1,”customer”,”sales”) So you can use it just like any built-in array function in spreadsheet. Please see here 2 examples: https://docs.google.com/spreadsheets/d/12TBoX2UI_Yu2MA2ZN3p9f-cZsySE4et1slwpgjZbSzw/edit#gid=422214765 The following is the … Read more

Convert matrix to 3-column table (‘reverse pivot’, ‘unpivot’, ‘flatten’, ‘normalize’)

To “reverse pivot”, “unpivot” or “flatten”: For Excel 2003: Activate any cell in your summary table and choose Data – PivotTable and PivotChart Report: For later versions access the Wizard with Alt+D, P. For Excel for Mac 2011, it’s ⌘+Alt+P (See here). Select Multiple consolidation ranges and click Next. In “Step 2a of 3”, choose … Read more

SQL Server : Columns to Rows

You can use the UNPIVOT function to convert the columns into rows: select id, entityId, indicatorname, indicatorvalue from yourtable unpivot ( indicatorvalue for indicatorname in (Indicator1, Indicator2, Indicator3) ) unpiv; Note, the datatypes of the columns you are unpivoting must be the same so you might have to convert the datatypes prior to applying the … Read more

tech