To summarize the answer on the mailing list thread: the problem was that the Ruby client wasn’t able to throttle its inserts, and Lucene memory usage does grow as large numbers of documents are added. I think there may also be an issue with commit frequency: it’s important to commit from time to time in order to flush newly added documents to disk. Is the OP still having the problem? If not, could you post the solution?