NumPy’s arrays are more compact than Python lists — a list of lists as you describe, in Python, would take at least 20 MB or so, while a NumPy 3D array with single-precision floats in the cells would fit in 4 MB. Access in reading and writing items is also faster with NumPy. Maybe you … Read more
OK, two steps to this – first is to write a function that does the translation you want – I’ve put an example together based on your pseudo-code: def label_race (row): if row[‘eri_hispanic’] == 1 : return ‘Hispanic’ if row[‘eri_afr_amer’] + row[‘eri_asian’] + row[‘eri_hawaiian’] + row[‘eri_nat_amer’] + row[‘eri_white’] > 1 : return ‘Two Or More’ … Read more
Using numpy.unique: import numpy a = numpy.array([0, 3, 0, 1, 0, 1, 2, 1, 0, 0, 0, 0, 1, 3, 4]) unique, counts = numpy.unique(a, return_counts=True) >>> dict(zip(unique, counts)) {0: 7, 1: 4, 2: 1, 3: 2, 4: 1} Non-numpy method using collections.Counter; import collections, numpy a = numpy.array([0, 3, 0, 1, 0, 1, 2, … Read more
I’ve tested all suggested methods plus np.array(list(map(f, x))) with perfplot (a small project of mine). Message #1: If you can use numpy’s native functions, do that. If the function you’re trying to vectorize already is vectorized (like the x**2 example in the original post), using that is much faster than anything else (note the log … Read more
Yes, given an array, array, and a value, item to search for, you can use np.where as: itemindex = numpy.where(array == item) The result is a tuple with first all the row indices, then all the column indices. For example, if an array is two dimensions and it contained your item at two locations then … Read more
To access column 0: >>> test[:, 0] array([1, 3, 5]) To access row 0: >>> test[0, :] array([1, 2]) This is covered in Section 1.4 (Indexing) of the NumPy reference. This is quick, at least in my experience. It’s certainly much quicker than accessing each element in a loop.
The criterion to satisfy for providing the new shape is that ‘The new shape should be compatible with the original shape’ numpy allow us to give one of new shape parameter as -1 (eg: (2,-1) or (-1,3) but not (-1, -1)). It simply means that it is an unknown dimension and we want numpy to … Read more
Use df.to_numpy() It’s better than df.values, here’s why.* It’s time to deprecate your usage of values and as_matrix(). pandas v0.24.0 introduced two new methods for obtaining NumPy arrays from pandas objects: to_numpy(), which is defined on Index, Series, and DataFrame objects, and array, which is defined on Index and Series objects only. If you visit … Read more
numpy.savetxt saves an array to a text file. import numpy a = numpy.asarray([ [1,2,3], [4,5,6], [7,8,9] ]) numpy.savetxt(“foo.csv”, a, delimiter=”,”)
Newer NumPy versions (1.8 and up) have a function called argpartition for this. To get the indices of the four largest elements, do >>> a = np.array([9, 4, 4, 3, 3, 9, 0, 4, 6, 0]) >>> a array([9, 4, 4, 3, 3, 9, 0, 4, 6, 0]) >>> ind = np.argpartition(a, -4)[-4:] >>> ind … Read more