Use tee (or equivalent) but limit max file size or rotate to new file

use split:

my_program | tee >(split -d -b 100000 -)

Or if you don’t want to see the output, you can directly pipe to split:

my_program | split -d -b 100000 -

As for the log rotation, there’s no tool in coreutils that does it automatically. You could create a symlink and periodically update it using a bash command:

while ((1)); do ln -fns target_log_name $(ls -t | head -1); sleep 1; done