They are different streams and are flushed at different times.
If you put
System.out.flush();
System.err.flush();
inside your loop, it will work as expected.
To clarify, output streams are cached so all the write goes into this memory buffer. After a period of quiet, they are actually written out.
You write to two buffers, then after a period of inactivity they both are flushed (one after the other).