Parallel.ForEach() is intended exactly for this kind of code.
On the other hand, ForAll() is intended to be used at the end of a (possibly complex) PLINQ query.
Because of that, I think Parallel.ForEach() is the better choice here.
In both cases, the current thread will be used to perform the computations (along with some threads from the thread pool) and the method will return only after all processing has been completed.