pandas

ImportError: IProgress not found. Please update jupyter and ipywidgets although it is installed

by

I tried everything you mentioned in a new environment using conda and I had another issue related to the version of ipywidgets (a bug found in Github with comments saying that got solved after using last version). I solved the problem I had installing last version of ipywidgets. Here is my process: Create a new … Read more

Read multiple parquet files in a folder and write to single csv file using python

by

I ran into this question looking to see if pandas can natively read partitioned parquet datasets. I have to say that the current answer is unnecessarily verbose (making it difficult to parse). I also imagine that it’s not particularly efficient to be constantly opening/closing file handles then scanning to the end of them depending on … Read more

How to reindex a MultiIndex dataframe

by

To get the B using reindex B.reindex( pd.MultiIndex.from_product([B.index.levels[0], A.index], names=[‘Bank’, ‘Curency’]),fill_value=0) Out[62]: Notional Bank Curency Bank_1 AUD 16 BRL 0 CAD 13 EUR 22 INR 0 Bank_2 AUD 24 BRL 0 CAD 20 EUR 17 INR 0 To get the A using concat pd.concat([A]*2,keys=B.index.levels[0]) Out[69]: AUD BRL CAD EUR INR Bank Bank_1 AUD 10 5 … Read more

Pandas .loc without KeyError

by

Update for @AlexLenail comment It’s a fair point that this will be slow for large lists. I did a little bit of more digging and found that the intersection method is available for Indexes and columns. I’m not sure about the algorithmic complexity but it’s much faster empirically. You can do something like this. good_keys … Read more

Pandas – possible to aggregate two columns using two different aggregations?

by

The agg method can accept a dict, in which case the keys indicate the column to which the function is applied: grouped.agg({‘numberA’:’sum’, ‘numberB’:’min’}) For example, import numpy as np import pandas as pd df = pd.DataFrame({‘A’: [‘foo’, ‘bar’, ‘foo’, ‘bar’, ‘foo’, ‘bar’, ‘foo’, ‘foo’], ‘B’: [‘one’, ‘one’, ‘two’, ‘three’, ‘two’, ‘two’, ‘one’, ‘three’], ‘number A’: … Read more