A simple Method for Tracking Code Progress in Python Notebooks

For a while now, I have often felt left in the dark when it comes to the progress of my code cells in notebooks. Therefore I wanted to find a clear and simple method for tracking how far through a for loop is.

The final output should tell me the current progress current run time and expected run time as shown in the first image:

 

 

Although there are perhaps nicer looking solutions using IPython widgets, the following method is the simplest I have found for achieving this. In the past, I have tended to simply print the current loop progress but doing this means that we will have a long list for each stage of the loop as shown below.

 

Instead, I would like to have the progress output in the same line. We can achieve this in three simple steps:

1) Import the ‘clear_output’ function from IPython.display

2) At the end of each loop, print the current progress

3) Remove the current progress at the start of the next loop step

This means we are essentially replacing the text in each stage of the loop and means we don’t have a long list we otherwise may have. This is shown in the code example below where the three parts mentioned are highlighted.

 

We can further improve this output slightly by attempting to estimate the expected total run time will be. This can be achieved by cross-referencing how long it has currently taken and how far the progress is. Of course, this is a really rough calculation and assumes the time taken will be linear but it at least gives a rough idea of how long it will take. The new calculations to do this are highlighted in the code below where we also exempt the expected time calculation for the first five percent to avoid inaccuracies.

 

I hope you find this interesting and useful, please let me know if you have any questions.

Thanks

Phil

Leave a Reply