First of all, in your example you make tests on a “data.frame” which is not fair for colMeans, apply and “[.data.frame” since they have an overhead: system.time(as.matrix(m)) #called by `colMeans` and `apply` # user system elapsed # 1.03 0.00 1.05 system.time(for(i in 1:ncol(m)) m[, i]) #in the `for` loop # user system elapsed # 12.93 … Read more
You simply use the apply() function: R> M <- matrix(1:6, nrow=3, byrow=TRUE) R> M [,1] [,2] [1,] 1 2 [2,] 3 4 [3,] 5 6 R> apply(M, 1, function(x) 2*x[1]+x[2]) [1] 4 10 16 R> This takes a matrix and applies a (silly) function to each row. You pass extra arguments to the function as … Read more
The apply functions in R don’t provide improved performance over other looping functions (e.g. for). One exception to this is lapply which can be a little faster because it does more work in C code than in R (see this question for an example of this). But in general, the rule is that you should … Read more
apply, the Convenience Function you Never Needed We start by addressing the questions in the OP, one by one. “If apply is so bad, then why is it in the API?” DataFrame.apply and Series.apply are convenience functions defined on DataFrame and Series object respectively. apply accepts any user defined function that applies a transformation/aggregation on … Read more
Try this: df <- df[,colSums(is.na(df))<nrow(df)]
You can return a Series from the applied function that contains the new data, preventing the need to iterate three times. Passing axis=1 to the apply function applies the function sizes to each row of the dataframe, returning a series to add to a new dataframe. This series, s, contains the new values, as well … Read more
Newer versions of pandas do allow you to pass extra arguments (see the new documentation). So now you can do: my_series.apply(your_function, args=(2,3,4), extra_kw=1) The positional arguments are added after the element of the series. For older version of pandas: The documentation explains this clearly. The apply method accepts a python function which should have a … Read more
Seems you forgot the ” of your string. In [43]: df[‘Value’] = df.apply(lambda row: my_test(row[‘a’], row[‘c’]), axis=1) In [44]: df Out[44]: a b c Value 0 -1.674308 foo 0.343801 0.044698 1 -2.163236 bar -2.046438 -0.116798 2 -0.199115 foo -0.458050 -0.199115 3 0.918646 bar -0.007185 -0.001006 4 1.336830 foo 0.534292 0.268245 5 0.976844 bar -0.773630 -0.570417 … Read more
OK, two steps to this – first is to write a function that does the translation you want – I’ve put an example together based on your pseudo-code: def label_race (row): if row[‘eri_hispanic’] == 1 : return ‘Hispanic’ if row[‘eri_afr_amer’] + row[‘eri_asian’] + row[‘eri_hawaiian’] + row[‘eri_nat_amer’] + row[‘eri_white’] > 1 : return ‘Two Or More’ … Read more