To eliminate duplicates from a list, you can maintain an auxiliary list and check against.
myList = ['Arise', 'But', 'It', 'Juliet', 'Who', 'already', 'and', 'and', 'and',
'breaks', 'east', 'envious', 'fair', 'grief', 'is', 'is', 'is', 'kill', 'light',
'moon', 'pale', 'sick', 'soft', 'sun', 'sun', 'the', 'the', 'the',
'through', 'what', 'window', 'with', 'yonder']
auxiliaryList = []
for word in myList:
if word not in auxiliaryList:
auxiliaryList.append(word)
output:
['Arise', 'But', 'It', 'Juliet', 'Who', 'already', 'and', 'breaks', 'east',
'envious', 'fair', 'grief', 'is', 'kill', 'light', 'moon', 'pale', 'sick',
'soft', 'sun', 'the', 'through', 'what', 'window', 'with', 'yonder']
This is very simple to comprehend and code is self explanatory. However, code simplicity comes on the expense of code efficiency as linear scans over a growing list makes a linear algorithm degrade to quadratic.
If the order is not important, you could use set()
A set object is an unordered collection of distinct hashable objects.
Hashability makes an object usable as a dictionary key and a set member, because these data structures use the hash value internally.
Since the average case for membership checking in a hash-table is O(1), using a set is more efficient.
auxiliaryList = list(set(myList))
output:
['and', 'envious', 'already', 'fair', 'is', 'through', 'pale', 'yonder',
'what', 'sun', 'Who', 'But', 'moon', 'window', 'sick', 'east', 'breaks',
'grief', 'with', 'light', 'It', 'Arise', 'kill', 'the', 'soft', 'Juliet']