Progress Bars in Jupyter

TQDM

I’ve found myself mapping functions onto large data sets lately, which sometimes takes minutes or even hours to complete. Using TQDM has been super helpful for giving me an idea of how long these operations can take. Of course the next step is to parallelize everything…

[Quick Bonus] Pip Install from a Notebook

import sys
!{sys.executable} -m pip install ipywidgets
# If using Jupyter Labs
!{sys.executable} -m jupyter labextension install @jupyter-widgets/jupyterlab-manager

Import TQDM and Setup

import pandas as pd
from tqdm.notebook import tqdm, trange
tqdm.pandas()

Now you can use progress bars

For loops:

for idx, row in tqdm(df.iterrows()):
  # Do something on each row

Apply on DataFrame:

def foo(row):
  # Do something on each row
df.progress_apply(foo, axis=1)