There are 3 possibilities:
- Increase
request.timeout.ms
– this is the time that Kafka will wait for whole batch to be ready in buffer. So in your case if there are less than 100 000 messages in buffer, timeout will occur. More info here: https://stackoverflow.com/a/34794261/2707179 - Decrease
batch-size
– related to previous point, it will send batches more often but they will include fewer messages. - Depending on message size, maybe your network cannot catch up with high load? Check if your throughput is not a bottleneck.