Skip to content

Tips and Tricks #2 Consider tqdm

  • blog
round life buoy

I’m writing a series of irregular short posts on some tips and tricks on using Geospatial Python. They may not be best practice, but I hope they are of help. I’ll try and give some context to when I’ve used the ‘tip’.

Use tqdm as a diagnostic tool to understand your iterables

You are going to need to install tqdm – follow guidance here

By all means use tqdm for your progress bars in your code – that is what it is meant for. This tip though is for those that want to understand why their code might be slow and secondly to visualise a loop within a loop. It might help you re think the way you’ve set your code up.

Ultimately all you need to do is wrap you iterable (in this example a for loop) with tqdm.

from tqdm import tqdm
import time

# just wrap any iterable with tqdm(iterable)

for i in tqdm (range(100)):

This code is a for loop of 100 steps and in each step it sleeps for 0.01 second. Its fast!

100%|██████████| 100/100 [00:01<00:00, 64.00it/s]

What happens when I do the same for a for loop within a for loop?

for i in tqdm (range(100), desc="Loading..."):
    # perhaps don't do this in real life!
    for k in tqdm(range(100)):


Crickey.. that is slow

Loading…: 3%|▎ | 3/100 [00:04<02:32, 1.57s/it]
0%| | 0/100 [00:00<?, ?it/s]
7%|▋ | 7/100 [00:00<00:01, 64.91it/s]
14%|█▍ | 14/100 [00:00<00:01, 64.80it/s]
21%|██ | 21/100 [00:00<00:01, 64.41it/s]
28%|██▊ | 28/100 [00:00<00:01, 64.19it/s]
35%|███▌ | 35/100 [00:00<00:01, 63.75it/s]
42%|████▏ | 42/100 [00:00<00:00, 63.86it/s]
49%|████▉ | 49/100 [00:00<00:00, 64.36it/s]
56%|█████▌ | 56/100 [00:00<00:00, 64.43it/s]
63%|██████▎ | 63/100 [00:00<00:00, 64.44it/s]
70%|███████ | 70/100 [00:01<00:00, 64.39it/s]
77%|███████▋ | 77/100 [00:01<00:00, 64.48it/s]
84%|████████▍ | 84/100 [00:01<00:00, 64.44it/s]
91%|█████████ | 91/100 [00:01<00:00, 64.21it/s]
100%|██████████| 100/100 [00:01<00:00, 64.22it/s]

or if I remove the tqdm part in the second loop

for i in tqdm (range(100), desc="Loading..."):
    # perhaps don't do this in real life!
    for k in range(100):

Loading…: 9%|▉ | 9/100 [00:14<02:22, 1.57s/it]

So consider using this in your for loops – almost certainly you will understand why your code is running slower, and hey if you’ve never used tqdm before perhaps I’ve just given you a progress bar you never knew you could have.

I’ve added comments where needed to the code here. I hope this has been of use.

I’ll be adding more tips and tricks

Image credit

I am a freelancer able to help you with your projects. I offer consultancy, training and writing. I’d be delighted to hear from you.

Feel free to connect or follow me; I am always keen to talk about Earth Observation.

I am @map_andrew on twitter