Update data.table v1.9.6+ now supports OP’s original attempt and the following answer is no longer necessary. You can use DT[order(-rank(x), y)]. x y v 1: c 1 7 2: c 3 8 3: c 6 9 4: b 1 1 5: b 3 2 6: b 6 3 7: a 1 4 8: a 3 … Read more
Use setcolorder(): library(data.table) x <- data.table(a = 1:3, b = 3:1, c = runif(3)) x # a b c # [1,] 1 3 0.2880365 # [2,] 2 2 0.7785115 # [3,] 3 1 0.3297416 setcolorder(x, c(“c”, “b”, “a”)) x # c b a # [1,] 0.2880365 3 1 # [2,] 0.7785115 2 2 # [3,] … Read more
rbindlist is an optimized version of do.call(rbind, list(…)), which is known for being slow when using rbind.data.frame Where does it really excel Some questions that show where rbindlist shines are Fast vectorized merge of list of data.frames by row Trouble converting long list of data.frames (~1 million) to single data.frame using do.call and ldply These … Read more
For versions of data.table >= 1.9.8, the following all just work: library(data.table) dt <- data.table(a = 1, b = 2, c = 3) # select single column by index dt[, 2] # b # 1: 2 # select multiple columns by index dt[, 2:3] # b c # 1: 2 3 # select single column … Read more
The reason pandas is faster is because I came up with a better algorithm, which is implemented very carefully using a fast hash table implementation – klib and in C/Cython to avoid the Python interpreter overhead for the non-vectorizable parts. The algorithm is described in some detail in my presentation: A look inside pandas design … Read more
This now works in v1.8.3 on R-Forge. Thanks for highlighting it! x <- data.table(a = 1:3, b = 1:6) f <- function(x) {list(“hi”, “hello”)} x[ , c(“col1”, “col2”) := f(), by = a][] # a b col1 col2 # 1: 1 1 hi hello # 2: 2 2 hi hello # 3: 3 3 hi … Read more
Good question. data.table can’t delete rows by reference yet. data.table can add and delete columns by reference since it over-allocates the vector of column pointers, as you know. The plan is to do something similar for rows and allow fast insert and delete. A row delete would use memmove in C to budge up the … Read more
Here’s a solution using data.table’s := operator, building on Andrie and Ramnath’s answers. require(data.table) # v1.6.6 require(gdata) # v2.8.2 set.seed(1) dt1 = create_dt(2e5, 200, 0.1) dim(dt1) [1] 200000 200 # more columns than Ramnath’s answer which had 5 not 200 f_andrie = function(dt) remove_na(dt) f_gdata = function(dt, un = 0) gdata::NAToUnknown(dt, un) f_dowle = function(dt) … Read more
Yes, in your formula, you can cbind the numeric variables to be aggregated: aggregate(cbind(x1, x2) ~ year + month, data = df1, sum, na.rm = TRUE) year month x1 x2 1 2000 1 7.862002 -7.469298 2 2001 1 276.758209 474.384252 3 2000 2 13.122369 -128.122613 … 23 2000 12 63.436507 449.794454 24 2001 12 999.472226 … Read more
.SD stands for something like “Subset of Data.table”. There’s no significance to the initial “.”, except that it makes it even more unlikely that there will be a clash with a user-defined column name. If this is your data.table: DT = data.table(x=rep(c(“a”,”b”,”c”),each=2), y=c(1,3), v=1:6) setkey(DT, y) DT # x y v # 1: a 1 … Read more