How to load 100 million records into MongoDB with Scala for performance testing?

Some tips :

  1. Do not index your collection before inserting, as inserts modify the index which is an overhead. Insert everything, then create index .

  2. instead of “save” , use mongoDB “batchinsert” which can insert many records in 1 operation. So have around 5000 documents inserted per batch.
    You will see remarkable performance gain .

    see the method#2 of insert here, it takes array of documents to insert instead of single document.
    Also see the discussion in this thread

    And if you want to benchmark more –

  3. This is just a guess, try using a capped collection of a predefined large size to store all your data. Capped collection without index has very good insertion performance.

Leave a Comment

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)