It’s not explained very well, but as far as I can tell, the server mode is synchronous per core, while the workstation mode is asynchronous.
In other words, the workstation mode is intended for a small number of long running applications that need consistent performance. The garbage collection tries to “stay out of the way” but, as a result, is less efficient on average.
The server mode is intended for applications where each “job” is relatively short lived and handled by a single core (edit: think multi threaded web server). The idea is that each “job” gets all the cpu power, and gets done quickly, but that occasionally the core stops handling requests and cleans up memory. So in this case the hope is that GC is more efficient on average, but the core is unavailable while its running, so the application needs to be able to adapt to that.
In your case it sounds like, because you have a single application whose threads are relatively coupled, you’re fitting better into the model expected by the first mode rather than the second.
But that’s all just after-the-fact justification. Measure your system’s performance (as ammoQ said, not your GC performance, but how well you application behaves) and use what you measure to be best.