vectorization

Vectorized way of calculating row-wise dot product two matrices with Scipy

by

Straightforward way to do that is: import numpy as np a=np.array([[1,2,3],[3,4,5]]) b=np.array([[1,2,3],[1,2,3]]) np.sum(a*b, axis=1) which avoids the python loop and is faster in cases like: def npsumdot(x, y): return np.sum(x*y, axis=1) def loopdot(x, y): result = np.empty((x.shape[0])) for i in range(x.shape[0]): result[i] = np.dot(x[i], y[i]) return result timeit npsumdot(np.random.rand(500000,50),np.random.rand(500000,50)) # 1 loops, best of 3: … Read more

numpy array TypeError: only integer scalar arrays can be converted to a scalar index

by

Short answer: [a[:,:j] for j in i] What you are trying to do is not a vectorizable operation. Wikipedia defines vectorization as a batch operation on a single array, instead of on individual scalars: In computer science, array programming languages (also known as vector or multidimensional languages) generalize operations on scalars to apply transparently to … Read more

Do any JVM’s JIT compilers generate code that uses vectorized floating point instructions?

by

So, basically, you want your code to run faster. JNI is the answer. I know you said it didn’t work for you, but let me show you that you are wrong. Here’s Dot.java: import java.nio.FloatBuffer; import org.bytedeco.javacpp.*; import org.bytedeco.javacpp.annotation.*; @Platform(include = “Dot.h”, compiler = “fastfpu”) public class Dot { static { Loader.load(); } static float[] … Read more

Why is vectorization, faster in general, than loops?

by

Vectorization (as the term is normally used) refers to SIMD (single instruction, multiple data) operation. That means, in essence, that one instruction carries out the same operation on a number of operands in parallel. For example, to multiply a vector of size N by a scalar, let’s call M the number of operands that size … Read more

Is the “*apply” family really not vectorized?

by

First of all, in your example you make tests on a “data.frame” which is not fair for colMeans, apply and “[.data.frame” since they have an overhead: system.time(as.matrix(m)) #called by `colMeans` and `apply` # user system elapsed # 1.03 0.00 1.05 system.time(for(i in 1:ncol(m)) m[, i]) #in the `for` loop # user system elapsed # 12.93 … Read more

Are for-loops in pandas really bad? When should I care?

by

TLDR; No, for loops are not blanket “bad”, at least, not always. It is probably more accurate to say that some vectorized operations are slower than iterating, versus saying that iteration is faster than some vectorized operations. Knowing when and why is key to getting the most performance out of your code. In a nutshell, … Read more

What is “vectorization”?

by

Many CPUs have “vector” or “SIMD” instruction sets which apply the same operation simultaneously to two, four, or more pieces of data. Modern x86 chips have the SSE instructions, many PPC chips have the “Altivec” instructions, and even some ARM chips have a vector instruction set, called NEON. “Vectorization” (simplified) is the process of rewriting … Read more