Confusion about pandas copy of slice of dataframe warning

izmir = pd.read_excel(filepath) izmir_lim = izmir[[‘Gender’,’Age’,’MC_OLD_M>=60′,’MC_OLD_F>=60′, ‘MC_OLD_M>18′,’MC_OLD_F>18′,’MC_OLD_18>M>5’, ‘MC_OLD_18>F>5′,’MC_OLD_M_Child<5′,’MC_OLD_F_Child<5’, ‘MC_OLD_M>0<=1′,’MC_OLD_F>0<=1′,’Date to Delivery’, ‘Date to insert’,’Date of Entery’]] izmir_lim is a view/copy of izmir. You subsequently attempt to assign to it. This is what is throwing the error. Use this instead: izmir_lim = izmir[[‘Gender’,’Age’,’MC_OLD_M>=60′,’MC_OLD_F>=60′, ‘MC_OLD_M>18′,’MC_OLD_F>18′,’MC_OLD_18>M>5’, ‘MC_OLD_18>F>5′,’MC_OLD_M_Child<5′,’MC_OLD_F_Child<5’, ‘MC_OLD_M>0<=1′,’MC_OLD_F>0<=1′,’Date to Delivery’, ‘Date to insert’,’Date of Entery’]].copy() Whenever you ‘create’ … Read more

How to find the line that is generating a Pandas SettingWithCopyWarning?

Set pd.options.mode.chained_assignment=”raise” This will throw an exception pointing to the line which triggers SettingWithCopyError. UPDATE: how to catch the error, and interrogate the stacktrace to get the actual offending lineno: import pandas as pd from inspect import currentframe, getframeinfo from pandas.core.common import SettingWithCopyError pd.options.mode.chained_assignment=”raise” df = pd.DataFrame({‘a’: [1, 2, 3], ‘b’: [4, 5, 6]}) df2 … Read more

Correct way to set value on a slice in pandas [duplicate]

This is a very common warning from pandas. It means you are writing in a copy slice, not the original data so it might not apply to the original columns due to confusing chained assignment. Please read this post. It has detailed discussion on this SettingWithCopyWarning. In your case I think you can try data.loc[data[‘name’] … Read more

What rules does Pandas use to generate a view vs a copy?

Here’s the rules, subsequent override: All operations generate a copy If inplace=True is provided, it will modify in-place; only some operations support this An indexer that sets, e.g. .loc/.iloc/.iat/.at will set inplace. An indexer that gets on a single-dtyped object is almost always a view (depending on the memory layout it may not be that’s … Read more

why should I make a copy of a data frame in pandas

This expands on Paul’s answer. In Pandas, indexing a DataFrame returns a reference to the initial DataFrame. Thus, changing the subset will change the initial DataFrame. Thus, you’d want to use the copy if you want to make sure the initial DataFrame shouldn’t change. Consider the following code: df = DataFrame({‘x’: [1,2]}) df_sub = df[0:1] … Read more