Excellent question. Here are a few upcoming features in Mesos to improve support for stateful services, and corresponding current workarounds. Persistent volumes (0.23): When launching a task, you can create a volume that exists outside of the task’s sandbox and will persist on the node even after the task dies/completes. When the task exits, its … Read more
Your summary is almost right but it does not reflect the essence of what mesos represents. The vision of mesosphere, the Company behind the project, is to create a “Datacenter Operating System” and the mesos is the kernel of it in analogy to the kernel of a normal OS. The API is not limited to … Read more
Disclaimer: I am the VP of Apache Aurora, and have been the tech lead of the Aurora team at Twitter for ~5 years. My likely-biased opinions are my own and do not necessarily represent those of Twitter or the ASF. Do these frameworks run anything that runs on Linux? For Marathon they state that it … Read more
Try running (just after the failure) dmesg. Do you see a line like this? Out of memory: Kill process 23747 (cc1plus) score 15 or sacrifice child Killed process 23747, UID 2243, (cc1plus) total-vm:214456kB, anon-rss:178936kB, file-rss:5908kB Most likely that is your problem. Running make -j 8 runs lots of process which use more memory. The problem … Read more
Spark Standalone Manager : A simple cluster manager included with Spark that makes it easy to set up a cluster. By default, each application uses all the available nodes in the cluster. A few benefits of YARN over Standalone & Mesos: YARN allows you to dynamically share and centrally configure the same pool of cluster … Read more