Not a complete solution – but a potential approach, based on the ‘guard’.
-
Start make with high ‘-j’ value.
-
Run a guard job that
a. Wait for thrashing (page fault/second above a limit), and high CPU utilization.
b. Find the jobs with the highest memory usage, suspend it.
c. Wait for some time to allow system to rebalance itself.
d. Repeat
If the approach looks good, can post sample implementation.