missing-data

How to get Python to gracefully format None and non-existing fields [duplicate]

by

The recommendation in PEP 3101 is to subclass Formatter: import string class PartialFormatter(string.Formatter): def __init__(self, missing=’~~’, bad_fmt=”!!”): self.missing, self.bad_fmt=missing, bad_fmt def get_field(self, field_name, args, kwargs): # Handle a key not found try: val=super(PartialFormatter, self).get_field(field_name, args, kwargs) # Python 3, ‘super().get_field(field_name, args, kwargs)’ works except (KeyError, AttributeError): val=None,field_name return val def format_field(self, value, spec): # handle … Read more

Best way to count the number of rows with missing values in a pandas DataFrame

by

For the second count I think just subtract the number of rows from the number of rows returned from dropna: In [14]: from numpy.random import randn df = pd.DataFrame(randn(5, 3), index=[‘a’, ‘c’, ‘e’, ‘f’, ‘h’], columns=[‘one’, ‘two’, ‘three’]) df = df.reindex([‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’, ‘g’, ‘h’]) df Out[14]: one two three a -0.209453 … Read more

How do I get a summary count of missing/NaN data by column in ‘pandas’?

by

Both describe and info report the count of non-missing values. In [1]: df = DataFrame(np.random.randn(10,2)) In [2]: df.iloc[3:6,0] = np.nan In [3]: df Out[3]: 0 1 0 -0.560342 1.862640 1 -1.237742 0.596384 2 0.603539 -1.561594 3 NaN 3.018954 4 NaN -0.046759 5 NaN 0.480158 6 0.113200 -0.911159 7 0.990895 0.612990 8 0.668534 -0.701769 9 -0.607247 … Read more