The only reason to use the log-likelihood instead of the plain old likelihood is mathematical convenience, because it lets you turn multiplication into addition. The plain old likelihood is P(parameters | data), i.e. assuming your data is fixed and you vary the parameters of your model. Maximizing this is one way to do parameter estimation … Read more
The distributions in scipy are coded in a generic way wrt two parameter location and scale so that location is the parameter (loc) which shifts the distribution to the left or right, while scale is the parameter which compresses or stretches the distribution. For the two parameter lognormal distribution, the “mean” and “std dev” correspond … Read more
If you have n data points (x[i], y[i], z[i]), compute the 3×3 symmetric matrix A whose entries are: sum_i x[i]*x[i], sum_i x[i]*y[i], sum_i x[i] sum_i x[i]*y[i], sum_i y[i]*y[i], sum_i y[i] sum_i x[i], sum_i y[i], n Also compute the 3 element vector b: {sum_i x[i]*z[i], sum_i y[i]*z[i], sum_i z[i]} Then solve Ax = b for the … Read more
What kind of linear regression? For something simple like least squares, I’d just program it myself: http://mathworld.wolfram.com/LeastSquaresFitting.html The math is not too hard to follow there, give it a shot for an hour or so and let me know if it’s too hard, I can try it. EDIT: Found someone that did it: http://dracoblue.net/dev/linear-least-squares-in-javascript/159/
Since this question has garnered a lot of views, I thought I’d post code for the final answer that I found, in part, by posting to the Eigen forums. The code uses Boost for the univariate normal and Eigen for matrix handling. It feels rather unorthodox, since it involves using the “internal” namespace, but it … Read more
You can get basic statistics by using “Show Log…” and then “Statistics” (Button at the bottom IIRC)
As @AbhranilDas mentioned, just use a linear method. There’s no need for a non-linear solver like scipy.optimize.lstsq. Typically, you’d use numpy.polyfit to fit a line to your data, but in this case you’ll need to do use numpy.linalg.lstsq directly, as you want to set the intercept to zero. As a quick example: import numpy as … Read more
You could use a groupby-agg operation: In [38]: result = df.groupby([‘a’], as_index=False).agg( {‘c’:[‘mean’,’std’],’b’:’first’, ‘d’:’first’}) and then rename and reorder the columns: In [39]: result.columns = [‘a’,’c’,’e’,’b’,’d’] In [40]: result.reindex(columns=sorted(result.columns)) Out[40]: a b c d e 0 Apple 3 4.5 7 0.707107 1 Banana 4 4.0 8 NaN 2 Cherry 7 1.0 3 NaN Pandas computes … Read more
I know it is an old question, but now it is possible to use random_score, with the following search query: { “size”: 1, “query”: { “function_score”: { “functions”: [ { “random_score”: { “seed”: “1477072619038” } } ] } } } For me it is very fast with about 2 million documents. I use current timestamp … Read more
You can use numpy’s built-in function var: import numpy as np results = [-14.82381293, -0.29423447, -13.56067979, -1.6288903, -0.31632439, 0.53459687, -1.34069996, -1.61042692, -4.03220519, -0.24332097] print(np.var(results)) This gives you 28.822364260579157 If – for whatever reason – you cannot use numpy and/or you don’t want to use a built-in function for it, you can also calculate it “by … Read more