A socket is the physical socket where the physical CPU capsules are placed. A normal PC only have one socket. Cores are the number of CPU-cores per CPU capsule. A modern standard CPU for a standard PC usually have two or four cores. And some CPUs can run more than one parallel thread per CPU-core. … Read more
You can work the other way around; rather than specifying which nodes to use, with the effect that each job is allocated all the 7 nodes, specify which nodes not to use: sbatch –exclude=myCluster[01-09] myScript.sh and Slurm will never allocate more than 7 nodes to your jobs. Make sure though that the cluster configuration allows … Read more
The locality level as far as I know indicates which type of access to data has been performed. When a node finishes all its work and its CPU become idle, Spark may decide to start other pending task that require obtaining data from other places. So ideally, all your tasks should be process local as … Read more
Disclosure: I’m a lead engineer on Kubernetes I think that Mesos and Kubernetes are largely aimed at solving similar problems of running clustered applications, they have different histories and different approaches to solving the problem. Mesos focuses its energy on very generic scheduling, and plugging in multiple different schedulers. This means that it enables systems … Read more