What is the fastest way to add data to a list without duplication in python (2.5)

You can use a set (in CPython since version 2.4) to efficiently look up duplicate values. If you really need an indexed system as well, you can use both a set and list.

Doing your lookups using a set will remove the overhead of if Item in List, but not that of List.index(Item)

Please note ItemNumber=List.index(Item) will be very inefficient to do after List.append(Item). You know the length of the list, so your index can be retrieved with ItemNumber = len(List)-1.

To completely remove the overhead of List.index (because that method will search through the list – very inefficient on larger sets), you can use a dict mapping Items back to their index.

I might rewrite it as follows:

# earlier in the program, NOT inside the loop
Dup = {}

# inside your loop to add items:
if Item in Dup:
    ItemNumber = Dup[Item]
else:
    List.append(Item)
    Dup[Item] = ItemNumber = len(List)-1

Leave a Comment

deneme bonusu veren sitelerbahis casinomakrobetceltabetpinbahispolobetpolobet girişpinbahis girişmakrobet girişpulibet girişmobilbahis girişkolaybet giriş