Loops

Learning Objectives
After completing this lesson, learners should be able to:

Use for loops to repeat operations multiple times

Running a script for multiple files

Motivation

In an imaging processing workflow you often apply the same operation to several images, several labels, etc. In order to avoid repeating the same code many times we can use control flow statements such as a for loop. Loops together with if clauses represent extremely useful tools when programming.

Concept map

graph TD A["Previous code"] --> Loop{"Check condition"} Loop --> |Condition is valid| RepeatChunk("Code chunk to be repeated") RepeatChunk --> Loop Loop --> |Condition is not valid| NextChunk("Next code to run")

Figure

In a control flow statement a piece of code is repeated (loop) as long as a specific condition is valid.

Activities

Open a script editor.
Open and run a script that contains several repeated operations and explain that you would like to write this part of code in a more simple way.
Explain the different elements of a numeric for loop:
- The loop header, the loop counter/variable, and body
- The initial and end condition
- How the counter is iterated (e.g. i++).
Using a print command show how the iterator changes
Take the starting script and modify it using a for loop

Show activity for:

ImageJ Macro, loop structure

// Before the loop
print ("Starting the process");


// We use for loop to print the processing status of 10 images
for (i = 0; i < 10 ; i++) {
    // we are using string concatenation to make this message
    print ("Index: "+i);
}

// After processing
print("Process completed");


// Self referencing variable
counter = 1;
print("counter " + counter);
counter = counter + 1;
print("counter " + counter);
// is equivalent to
counter = 1;
print("counter " + counter);
counter++;
print("counter " + counter);

// You can also have a loop in a loop

for (j = 0; j < 10 ; j++) {

    for (i = 0; i < 10 ; i++) {
        // we are using string concatenation to make this message
        print ("Index big loop "+j + "; index small loop " + i);
    }
}

ImageJ Macro, example no loop

/*
 * Measure distance of all labels to the first label 
 * 
 * Requirements: 
 * - IJPB-plugins update site
 */
 
// This is what we would do when recording a macro

run("Close All");
open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_16bit_labels__nuclei.tif");
rename("labels");
run("Duplicate...", "title=binary");
run("Manual Threshold...", "min=1 max=1");
run("Convert to Mask");
run("Invert");
run("Chamfer Distance Map", "distances=[Chessknight (5,7,11)] output=[16 bits] normalize");
run("Intensity Measurements 2D/3D", "input=binary-dist labels=labels mean max min");
Table.rename("binary-dist-intensity-measurements", "dist_label"+labelID);

ImageJ Macro, example with loop

/*
 * Measure distance of all labels to the all labels
 * 
 * Requirements: 
 * - IJPB-plugins update site
 */
// Macro to measure closest, furthest and mean distance of labels to each other
// This is a version with a loop
run("Close All");
open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_16bit_labels__nuclei.tif");
rename("labels");
// Explain how to find maximal value of loop
getRawStatistics(nPixels, mean, min, max, std, histogram);

for (i = 0; i < max; i++) {
	run("Duplicate...", "title=binary");
	labelID = i+1;
	run("Manual Threshold...", "min="+labelID+ " max="+ labelID);
	run("Convert to Mask");
	run("Invert");
	run("Chamfer Distance Map", "distances=[Chessknight (5,7,11)] output=[16 bits] normalize");
	run("Intensity Measurements 2D/3D", "input=binary-dist labels=labels mean max min");
	close("binary");
	close("binary-dist");
	Table.rename("binary-dist-intensity-measurements", "dist_label"+labelID);
}

Python, for loop

# %%
# Create a bunch of example images to simulate a typical analysis problem
# np.random.randint(lower,upper,size): generates random integers from lower to upper
# .reshape: reshapes a np.array to new dimensions
import numpy as np
image1 = np.random.randint(0,255,100).reshape((10,10))
image2 = np.random.randint(0,255,100).reshape((10,10))
image3 = np.random.randint(0,255,100).reshape((10,10))
image4 = np.random.randint(0,255,100).reshape((10,10))
image5 = np.random.randint(0,255,100).reshape((10,10))
image6 = np.random.randint(0,255,100).reshape((10,10))
image7 = np.random.randint(0,255,100).reshape((10,10))
image8 = np.random.randint(0,255,100).reshape((10,10))
image9 = np.random.randint(0,255,100).reshape((10,10))
image10 = np.random.randint(0,255,100).reshape((10,10))

# Calculate the mean of every image
# Oberserve that it is very tedious and error prone
print(f'Image {1} has an avg intensity of {image1.mean()}.')
print(f'Image {2} has an avg intensity of {image2.mean()}.')
print(f'Image {3} has an avg intensity of {image3.mean()}.')
print(f'Image {4} has an avg intensity of {image4.mean()}.')
print(f'Image {5} has an avg intensity of {image5.mean()}.')
print(f'Image {6} has an avg intensity of {image6.mean()}.')
print(f'Image {7} has an avg intensity of {image7.mean()}.')
print(f'Image {8} has an avg intensity of {image8.mean()}.')
print(f'Image {9} has an avg intensity of {image9.mean()}.')
print(f'Image {10} has an avg intensity of {image10.mean()}.')

# %%
# Create a for loop
# Typical notation in C-style: for i=0, i<10, i++ {do something}
# For loop in python
# Observe the difference in notation
for i in [0,1,2,3,4,5,6,7,8,9]:
    print(i)

# use range
# Observe that range is actually a generator producing values on the fly
print(range(10))
print(type(range(10)))
for i in range(10):
    print(i)

# %%
# Anything iterable can be iterated over with a for loop
# Observe that:
# - list content don't matter
# - that the iterator will always be overwritten by the next item
for i in ['a','b','c',1,2,3,[1,2,3],1.5,2.7]:
    print(f'"i" has the value "{i}" and type "{type(i)}"')

# %%
# Use a for loop for the example in the beginning
# First pack images into a list for looping over
image_list = [image1,image2,image3,image4,image5,image6,image7,image8,image9,image10]

# Loop over images and calculate the mean
for image in image_list:
    print(f'Avg intensity: {image.mean()}')

# Excersize: Modify the loop to calculate the standard deviation
for image in image_list:
    print(f'Intensity standard deviation: {image.std()}')

#%%
# Iterate with an index
number_of_images = len(image_list)
print(number_of_images)
for i in range(number_of_images):
    print(f'Image {i} has an avg intensity of {image_list[i].mean()}.')

# Iterate over two or more list (with the same length)
for i,image in zip(range(number_of_images),image_list):
    print(f'Image {i} has an avg intensity of {image.mean()}.')

# Iterate with additional index
for i,image in enumerate(image_list):
    print(f'Image {i} has an avg intensity of {image.mean()}.')

# %%
# For loops but advanced

# %%
# list comprehension
squares = []
for i in range(10):
    squares.append(i**2)
print(squares)
# use list comprehension
squares = [i**2 for i in range(10)]
print(squares)

# usefull for creating an filepath iterable
from pathlib import Path
[file for file in Path().cwd().iterdir()]

# %%
# for loop with if
for x in range(10):
    if x != 0:
        print(f'X: {x}, 1/X: {1/x}')

# for loop with if and else
for x in range(10):
    if x == 0:
        print(f'X: {x}, 1/X: Division by {x} not defined.')
    else:
        print(f'X: {x}, 1/X: {1/x}')

# %%
# for loop with continue and break
# Usage:
# - skip files/data for processing
# - debugging
# continue
for x in range(10):
    if x==0:
        continue
    print(f'X: {x}, 1/X: {1/x}')

# break
for x in range(10):
    if x==3:
        break
    print(x)

# %%
# Nested loops
for x in range(1,4):
    for y in range(1,4):
        print(f'X: {x}, Y: {y}, X*Y: {x*y}')

# Nested lists
for sublist in [[1,2,3],[4,5,6],[7,8,9]]:
    for i in sublist:
        print(i)

# np.array
# Observe that you iterate only over the first dimension when using simple loop
array = np.random.randint(0,10,100).reshape(10,10)
print(array.shape)
for i in array:
    print(i)

for row in array:
    for i in row:
        print(i)

# %%
# Looping over pandas DataFrame
# Observe that
# - looping with iterrows always gives an index
# - the second variable will always be a pandas Series object
import pandas as pd
df = pd.DataFrame({
    'a':[1,2,3],
    'b':[4,5,6]})
print(df)
for i,row in df.iterrows():
    print(type(row))
    value_a = row['a']
    value_b = row['b']
    print(f'Row index {i}\nColumn value "a": {value_a} and Column value "b": {value_b}')

Exercises

Multiple erosion

Open the binary image xy_8bit_binary__nuclei.tif
Create a macro (or use a pre-defined one) to erode and find boundary of the binary objects
Modify the macro so that you can perform an arbitrary number of erosions and find their boundary

Show exercise/solution for:

ImageJ Macro, Multiple erosion
Download the script script_for_loop_erodeband_noloop.ijm. The goal is to perform a series of binary-erosions and compute the outline of these objects.

Create a variables maxErode that sets the number of erode operations

Create a for loop to perform maxErode operations, change the name of the image accordingly

(Advanced) Create a second for loop that uses maxErode as counter. Vary maxErode from 1 to 50

A solution can be found in script_for_loop_erodeband_withloop.ijm

Assessment

Explanations

For loop

A for loop occurs by iterating over a loop variable defined in a loop header. You use for loops when you know the number of iterations to execute.

While loop

While loop does not have a fixed number of iterations. Typically the header contains a condition that is computed within the body of the loop. TODO.

Follow-up material

Recommended follow-up modules:

Learn more:

ImageJ macro loops

Image processing with python, search for loop keyword to see examples