Should I always use Parallel.Foreach because more threads MUST speed up everything?

No, it doesn’t make sense for every foreach. Some reasons:

• Your code may not actually be parallelizable. For example, if you’re using the “results so far” for the next iteration and the order is important)
• If you’re aggregating (e.g. summing values) then there are ways of using Parallel.ForEach for this, but you shouldn’t just do it blindly
• If your work will complete very fast anyway, there’s no benefit, and it may well slow things down

Basically nothing in threading should be done blindly. Think about where it actually makes sense to parallelize. Oh, and measure the impact to make sure the benefit is worth the added complexity. (It will be harder for things like debugging.) TPL is great, but it’s no free lunch.