You will lose time switching between reads and writes, and the limit of the chunk size is AFAIK only the limit of what you can store in memory. So as long as you aren’t very concerned about keeping memory usage down, go ahead and specify a large chunk size, such as 1 MB (e.g. 1024 * 1024) or even 10 MB. Chunk sizes in the 1024 byte range (or even smaller, as it sounds like you’ve tested much smaller sizes) will slow the process down substantially.
For a very heavy-duty situation where you want to get as much performance as possible out of your code, you could look at the io module for buffering etc. But I think increasing the chunk size by a factor of 1000 or 10000 or so will probably get you most of the way there.