Note for pandas versions < 0.20.0: I will be using df.melt(…) for my examples, but you will need to use pd.melt(df, …) instead. Documentation references: Most of the solutions here would be used with melt, so to know the method melt, see the documentation explanation. Unpivot a DataFrame from wide to long format, optionally leaving … Read more
This can be done with pd.melt(): # value_name is ‘value’ by default, but setting it here to make it clear pd.melt(x, id_vars=[‘farm’, ‘fruit’], var_name=”year”, value_name=”value”) Result: farm fruit year value 0 A apple 2014 10 1 B apple 2014 12 2 A pear 2014 6 3 B pear 2014 8 4 A apple 2015 11 … Read more
You can melt your dataframe: >>> keys = [c for c in df if c.startswith(‘key.’)] >>> pd.melt(df, id_vars=”topic”, value_vars=keys, value_name=”key”) topic variable key 0 8 key.0 abc 1 9 key.0 xab 2 8 key.1 def 3 9 key.1 xcd 4 8 key.2 ghi 5 9 key.2 xef It also gives you the source of the … Read more
Use pandas.melt or pandas.DataFrame.melt to transform from wide to long: df = pd.DataFrame({ ‘date’ : [’05/03′, ’06/03′, ’07/03′, ’08/03′], ‘AA’ : [1, 4, 7, 5], ‘BB’ : [2, 5, 8, 7], ‘CC’ : [3, 6, 9, 1] }).set_index(‘date’) df AA BB CC date 05/03 1 2 3 06/03 4 5 6 07/03 7 8 9 … Read more
First I convert lower values of df to NaN by where and numpy.triu and then stack, reset_index and set column names: import numpy as np print df a b c a 1.0 0.5 0.3 b 0.5 1.0 0.4 c 0.3 0.4 1.0 print np.triu(np.ones(df.shape)).astype(np.bool) [[ True True True] [False True True] [False False True]] df … Read more
Spark >= 3.4 In Spark 3.4 or later you can use built-in melt method (sdf .melt( ids=[‘A’], values=[‘B’, ‘C’], variableColumnName=”variable”, valueColumnName=”value”) .show()) +—+——–+—–+ | A|variable|value| +—+——–+—–+ | a| B| 1| | a| C| 2| | b| B| 3| | b| C| 4| | c| B| 5| | c| C| 6| +—+——–+—–+ This method is available … Read more
there are a few ways; using .pivot: >>> origin.pivot(index=’label’, columns=”type”)[‘value’] type a b c label x 1 2 3 y 4 5 6 z 7 8 9 [3 rows x 3 columns] using pivot_table: >>> origin.pivot_table(values=”value”, index=’label’, columns=”type”) value type a b c label x 1 2 3 y 4 5 6 z 7 8 … Read more