tqdm
Using the tqdm
package, a fast and versatile progress bar utility
pip install tqdm
from tqdm import tqdm
def process(token):
return token['text']
l1 = [{'text': k} for k in range(5000)]
l2 = [process(token) for token in tqdm(l1)]
100%|███████████████████████████████████| 5000/5000 [00:00<00:00, 2326807.94it/s]
No requirement
1/ Use a side function
def report(index):
if index % 1000 == 0:
print(index)
def process(token, index, report=None):
if report:
report(index)
return token['text']
l1 = [{'text': k} for k in range(5000)]
l2 = [process(token, i, report) for i, token in enumerate(l1)]
2/ Use and
and or
statements
def process(token):
return token['text']
l1 = [{'text': k} for k in range(5000)]
l2 = [(i % 1000 == 0 and print(i)) or process(token) for i, token in enumerate(l1)]
3/ Use both
def process(token):
return token['text']
def report(i):
i % 1000 == 0 and print(i)
l1 = [{'text': k} for k in range(5000)]
l2 = [report(i) or process(token) for i, token in enumerate(l1)]
All 3 methods print:
0
1000
2000
3000
4000
How 2 works
i % 1000 == 0 and print(i)
:and
only checks the second statement if the first one isTrue
so only prints wheni % 1000 == 0
or process(token)
:or
always checks both statements, but returns the first one which evals toTrue
.- If
i % 1000 != 0
then the first statement isFalse
andprocess(token)
is added to the list. - Else, then the first statement is
None
(becauseprint
returnsNone
) and likewise, theor
statement addsprocess(token)
to the list
- If
How 3 works
Similarly as 2, because report(i)
does not return
anything, it evals to None
and or
adds process(token)
to the list