Easiest way to install Python dependencies on Spark executor nodes?

Actually having actually tried it, I think the link I posted as a comment doesn’t do exactly what you want with dependencies. What you are quite reasonably asking for is a way to have Spark play nicely with setuptools and pip regarding installing dependencies. It blows my mind that this isn’t supported better in Spark. … Read more

Creating Haskell shared libraries on OS X

It is possible to create working shared libraries on 64-bit OS X, with the latest Haskell Platform release (2012.4 64bit) The invocation line works for me: ghc -O2 –make \ -no-hs-main -optl ‘-shared’ -optc ‘-DMODULE=Test’ \ -o libTest.so Test.hs module_init.c module_init.c should be something like: #define CAT(a,b) XCAT(a,b) #define XCAT(a,b) a ## b #define STR(a) … Read more

Stripping linux shared libraries

So the solution we have for now is as follows: test.cpp #include <cmath> #include <vector> #include <typeinfo> struct private_struct { float f; }; float private_function(float f) { return std::abs(f); } void other_private_function() { std::vector<private_struct> f(1); } extern “C” void __attribute__ ((visibility (“default”))) public_function2() { other_private_function(); } extern “C” float __attribute__ ((visibility (“default”))) public_function1(float f) { … Read more

Shared library in Go?

This is possible now using -linkshared flag What you need to do is to first run this command: go install -buildmode=shared -linkshared std (Above code makes all common packages shareable!) then go install -buildmode=shared -linkshared userownpackage finally when compiling your code you need to run: go build -linkshared yourprogram What the above those is now … Read more

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)