What’s the best way to sum all values in a Pandas dataframe? [duplicate]
Updated for Pandas 0.24+ df.to_numpy().sum() Prior to Pandas 0.24+ df.values Is the underlying numpy array df.values.sum() Is the numpy sum method and is faster
Updated for Pandas 0.24+ df.to_numpy().sum() Prior to Pandas 0.24+ df.values Is the underlying numpy array df.values.sum() Is the numpy sum method and is faster
I may be wrong but from what I understand conversions.id is the primary key of your table conversions stats.id is the primary key of your table stats Thus for each conversions.id you have at most one links.id impacted. You request is a bit like doing the cartesian product of 2 sets : [clicks] SELECT * … Read more
I think the simplest way: df.groupBy().sum().collect() will return a list. In your example: In [9]: df.groupBy().sum().collect()[0][0] Out[9]: 130
sum is quite fast, but sum isn’t the cause of the slowdown. Three primary factors contribute to the slowdown: The use of a generator expression causes overhead for constantly pausing and resuming the generator. Your generator version adds unconditionally instead of only when the digit is even. This is more expensive when the digit is … Read more
How about: SELECT COALESCE(sum(num), 0) AS val FROM tab WHERE descr LIKE “%greetings%”; The COALESCE function basically says “return the first parameter, unless it’s null in which case return the second parameter” – It’s quite handy in these scenarios.
How about this: UPDATE p SET p.extrasPrice = t.sumPrice FROM BookingPitches AS p INNER JOIN ( SELECT PitchID, SUM(Price) sumPrice FROM BookingPitchExtras WHERE [required] = 1 GROUP BY PitchID ) t ON t.PitchID = p.ID WHERE p.bookingID = 1
if you want to operate with list of numbers it is better to use NumPy arrays: import numpy a = [1, 1, 1 ,1, 1] ar = numpy.array(a) print ar + 2 gives [3, 3, 3, 3, 3]
As numpy.convolve is pretty slow, those who need a fast performing solution might prefer an easier to understand cumsum approach. Here is the code: cumsum_vec = numpy.cumsum(numpy.insert(data, 0, 0)) ma_vec = (cumsum_vec[window_width:] – cumsum_vec[:-window_width]) / window_width where data contains your data, and ma_vec will contain moving averages of window_width length. On average, cumsum is about … Read more
@Ney @hpaulj is correct, you need to experiment, but I suspect you don’t realize that summation for some arrays can occur along axes. Observe the following which reading the documentation >>> a array([[0, 0, 0], [0, 1, 0], [0, 2, 0], [1, 0, 0], [1, 1, 0]]) >>> np.sum(a, keepdims=True) array([[6]]) >>> np.sum(a, keepdims=False) 6 … Read more
Python tries to discourage you from “summing” strings. You’re supposed to join them: “”.join(list_of_strings) It’s a lot faster, and uses much less memory. A quick benchmark: $ python -m timeit -s ‘import operator; strings = [“a”]*10000’ ‘r = reduce(operator.add, strings)’ 100 loops, best of 3: 8.46 msec per loop $ python -m timeit -s ‘import … Read more