After completing this lesson, learners should be able to:
Understand how to design morphological filters using rank filters
Execute morphological filters on binary or label images and understand the output
Motivation
Morphological filters (MFs) are used to clean up segmentation masks and achieve a change in morphology and/or size of the objects. For example, MFs are used to remove wrongly assigned foreground pixels, separate touching objects, or identify objects boundaries.
Concept map
graph TD
subgraph opening
erode("Erode (min)") --> dilate("Dilate (max)")
end
subgraph closing
dilate2("Dilate (max)") --> erode2("Erode (min)")
end
subgraph rank operations
any("...")
end
BI("Binary/label image") --> SE("structuring element")
SE .-> erode
SE .-> dilate2
SE .-> any
dilate .-> BIM
erode2 .-> BIM
any .-> BIM("Modified binary/label image")
Figure
A dilation and erosion using a 3x3 structuring element (left side). Morphological filters applied in series, e.g. opening and closing, can achieve very useful results (right side).
Perform erosion followed by dilation - opening. Explains it effects in removing thin structures, smoothing borders. If applicable show that opening runs as single command.
Perform dilation followed by erosion - closing. Explains it effects on filling small holes, connecting gaps. If applicable show that opening runs as single command.
Subtract eroded image from binary image and discuss the results (Internal Gradient)
If applicable show where the morphological gradient runs as a single command
Show activity for:
ImageJ Macro & GUI: Dilation and erosion
run("Close All");//Make sure black background in Process > Binary > Options is set to truesetOption("black background",true);open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__two_spots_different_size.tif");// Image > Renamerename("binary");run("Set... ","zoom=400 x=29 y=25");// Erosion, use default binary IJ binary operations// It is a radius 1 squared SE, i.e. 3x3 SE// Image > Duplicate, name it erodedrun("Duplicate...","title=eroded");//Process > Binary > Eroderun("Erode");run("Set... ","zoom=400 x=29 y=25");// Apply second erosion will remove small dot// Image > Duplicate, name it eroded_twicerun("Duplicate...","title=eroded_twice");// Process > Binary > Eroderun("Erode");run("Set... ","zoom=400 x=29 y=25");// Use MorpholibJ and a radius 2, i.e. 5x5 squared structuring element// This corresponds to 2 sequntial 3x3 erosionsselectWindow("binary");// Plugins > MorpholibJ > Filtering > Morphological Filters // Select Erosion, Element Square run("Morphological Filters","operation=Erosion element=Square radius=2");rename("erosion_radius2");run("Set... ","zoom=400 x=29 y=25");// Dilation, use MorpholibJselectWindow("binary");// Plugins > MorpholibJ > Filtering > Morphological Filters// Use MorpholibJ and a radius 2, i.e. 5x5 squared structuring elementrun("Morphological Filters","operation=Dilation element=Square radius=1");rename("dilation_radius1");run("Set... ","zoom=400 x=29 y=25");// Dilation, use MorpholibJselectWindow("binary");// Plugins > MorpholibJ > Filtering > Morphological Filters// Use MorpholibJ and a radius 2, i.e. 5x5 squared structuring elementrun("Morphological Filters","operation=Dilation element=Square radius=2");rename("dilation_radius2");run("Set... ","zoom=400 x=29 y=25");run("Tile")
ImageJ Macro & GUI: Closing and opening
run("Close All");//Make sure black background in Process > Binary > Options is set to truesetOption("black background",true);open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__for_open_and_close.tif")rename("binary");run("Set... ","zoom=400 x=29 y=25");// Opening, use default binary IJ binary operations in sequencerun("Duplicate...","title=opening");//Process > Binary > Eroderun("Erode");//Process > Binary > Dilaterun("Dilate");run("Set... ","zoom=400 x=29 y=25");// See how the thin structure disappear// Opening, use default binary IJ binary operationsselectWindow("binary");run("Duplicate...","title=opening2");//Process > Binary > Openrun("Open");run("Set... ","zoom=400 x=29 y=25");// Opening, use MorpholibJ, try different radiiselectWindow("binary");// Plugins > MorpholibJ > Filtering > Morphological Filtersrun("Morphological Filters","operation=Opening element=Square radius=1");rename("binary-Opening_radius=1");run("Set... ","zoom=400 x=29 y=25");selectWindow("binary");// Plugins > MorpholibJ > Filtering > Morphological Filtersrun("Morphological Filters","operation=Opening element=Square radius=3");rename("binary-Opening_radius=3");run("Set... ","zoom=400 x=29 y=25");// see how also the small blob disappear, side of large blob are deformed// Closing, use MorpholibJselectWindow("binary");// Plugins > MorpholibJ > Filtering > Morphological Filtersrun("Morphological Filters","operation=Closing element=Square radius=1");rename("binary-Closing_radius=1");run("Set... ","zoom=400 x=29 y=25");// Fill the hole in big blob// Closes the gaprun("Tile")
ImageJ Macro & GUI: Internal Gradient
run("Close All");//Make sure black background in Process > Binary > Options is set to truesetOption("black background",true);open("https://raw.githubusercontent.com/NEUBIAS/training-resources/master/image_data/xy_8bit_binary__h2b.tif");rename("binary");// Internal gradient is the original - eroded image// Shown as step by step process// Erode image// Plugins > MorpholibJ > Filtering > Morphological Filters// Operation = Erosion// Element = Square// Radius (in pixels) = 1run("Morphological Filters","operation=Erosion element=Square radius=1");rename("eroded")// Process > Image Calculator ... // image1 = binary// operation = subtract// image2 = binary-Erosion// [x] Create enew windowimageCalculator("Subtract create","binary","eroded");rename("internal_gradient");// Add internal_gradient as overlayselectImage("binary");// Image > Overlay > Add..run("Add Image...","image=internal_gradient x=0 y=0 opacity=50");//Internal gradient with MorpholibJselectWindow("binary");// Plugins > MorpholibJ > Filtering > Morphological Filters// Operation = Erosion// Element = Square// Radius (in pixels) = 1run("Morphological Filters","operation=[Internal Gradient] element=Square radius=1");run("Tile")
skimage napari
#######################################################
## To follow along you need to complete the tool
## installation activity for skimage napari.
#######################################################
#%%
# Import python packages.
fromOpenIJTIFFimportopen_ij_tifffromnapari.viewerimportViewerfromskimage.morphologyimportsquare,diskfromskimage.morphologyimporterosion,dilationfromskimage.morphologyimportopening,closing#%%
# download and read tif file
fpath="https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__two_spots_different_size.tif"image_2spots,_,_,_=open_ij_tiff(fpath)# Create a napari_viewer and visualize image and labels.
napari_viewer=Viewer()napari_viewer.add_image(image_2spots,name='image')#%%
# Perform erosion and dilation with default structuring element (cross, connectivity=1).
eroded=erosion(image_2spots)dilated=dilation(image_2spots)#%%
# Now try with connectivity 2 (square).
square3=square(3)print(square3)eroded_square3=erosion(image_2spots,square3)dilated_square3=dilation(image_2spots,square3)#%%
# add images to the viewer and inspect the results
napari_viewer.add_image(eroded,name='eroded1')napari_viewer.add_image(dilated,name='dilated1')napari_viewer.add_image(eroded_square3,name='eroded3')napari_viewer.add_image(dilated_square3,name='dilated3')# Now try with bigger square (e.g. square(5))
# or a different structuring element (e.g. disk(1))
# Also refer to:
# https://scikit-image.org/docs/stable/api/skimage.morphology.html
#%%
# Explore opening and closing combining erosion and dilation.
# Verify that they give the same results.
fpath="https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__for_open_and_close.tif"image_structures,_,_,_=open_ij_tiff(fpath)#%%
# Opening: remove thin structures, smoothen objects.
opening_manual=dilation(erosion(image_structures,square(3)),square(3))opening_1step=opening(image_structures,square(3))print((opening_manual==opening_1step).all())#%%
# Closing: fill holes, connect gaps.
closing_manual=erosion(dilation(image_structures,square(3)),square(3))closing_1step=closing(image_structures,square(3))print((closing_manual==closing_1step).all())#%%
# Try using the default structuring element (disk(1)).
# Explain what has changed and why.
closing_disk=closing(image_structures,disk(1))#%%
# Create a napari_viewer and visualize images.
napari_viewer=Viewer()napari_viewer.add_image(image_structures,name='image')napari_viewer.add_image(opening_1step,name='opening')napari_viewer.add_image(closing_1step,name='closing')napari_viewer.add_image(closing_disk,name='closing_disk')#%%
# Explore internal gradient.
fpath="https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__h2b.tif"image,_,_,_=open_ij_tiff(fpath)# Internal gradient is the difference between the image and the eroded version of it.
eroded=erosion(image)internal_gradient=image-eroded#%%
# Create a napari_viewer and visualize images.
napari_viewer=Viewer()napari_viewer.add_image(image,name='image')napari_viewer.add_image(internal_gradient,name='internal_gradient')# The internal gradient represents the edge of the object.
# Discuss when and how this can be useful.
Exercises
Measure intensity on the nuclear membrane
In image xyc_16bit__nup__nuclei.tif we would like to measure the intensity along the nuclear membrane (channel 1) using the information from the DNA (channel 2). We designed two exercises that provide a workflow using morphological filters.
Clean up segmentation
Use a combination of opening and closing operations to improve the segmentation of the DNA channel xy_8bit_binary__nuclei_noisy.tif.
The goal is to achieve something like xy_8bit_binary__nuclei.tif that can be used for further processing and identification of membrane regions.
Define nuclear rim
Use morphological filtering to define an inner rim of width 3 pixels using the label mask: xy_8bit_labels__nuclei.tif
(Optional) Measure the mean and total intensity in the first channel of xyc_16bit__nup__nuclei.tif using the modified labels masks.
run("Close All");open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyc_16bit__nup_nuclei/xy_8bit_labels__nuclei.tif");run("Morphological Filters","operation=[Internal Gradient] element=Square radius=3");rename("rim");open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyc_16bit__nup_nuclei.tif");run("Duplicate...","title=Ch1 duplicate channels=1");run("Intensity Measurements 2D/3D","input=Ch1 labels=rim mean stddev max min median numberofvoxels");
Assessment
Fill in the blanks
Using those words fill in the blanks: closing, opening, min, shrinks, decreases, enlarges, max.
An erosion _____ objects in a binary image.
An erosion in a binary image _____ the number of foreground pixels.
A dilation _____ objects in a binary image.
An erosion of a binary image correspods to a ___ rank operation.
An dilation of a binary image correspods to a ___ rank operation.
A dilation followed by an erosion is called ___.
An erosion followed by a dilation is called ___ .
Solution
shrinks
decreases
enlarges
min
max
closing
opening
True of false?
Discuss with your neighbour!
Morphological openings on binary images never decrease the number of foreground pixels.
Morphological closings on binary images never decreases the number of foreground pixels.
Performing a morphological closing twice in a row does not make sense, because the second closing does not further change the image.
Performing a morphological closing with radius 2 element is equivalent to two subsequent closing operation with radius 1.
Solution
False
True
True
False
Explanations
Rank filters
In the region defined by the structuring element, pixel elements are ranked/sorted according to their values. The pixel in the filtered image is replaced with the corresponding sorted pixel (smallest = min, greatest = max, median ). See also Median filter. Morphological filters corresponds to one or several rank filters applied to an image.
Morphological filters on binary images
A typical application of these filters is to refine segmentation results. A max-filter is called dilation whereas a min-filter is called erosion. Often rank filters are applied in a sequence. We refer to a closing operation as a max-filter followed by a min-filter of the same size. An opening operation is the inverse, a min-filter followed by a max-filter.
Opening operations will:
Remove small/thin objects which extent is below the size of the structuring element
Smooth border of an object
Closing operations:
Fill small holes below the size of the structuring element
Can connect gaps
Image subtraction using eroded/dilated images allows to identify the boundary of objects and is referred to morphological gradients:
Internal gradient: original - eroded
External gradient: dilated - original
(Symmetric) gradient: dilated - eroded
Fill holes operation is a slightly more complex morphological operation. It is used to identify background pixels surrounded by foreground pixels and change their value to foreground. Algorithmically there are several ways to achieve this.
Morphological filters on label images
Morphological filters work also on label images. If the objects are not touching this will achieve the expected result for each label. However, when objects touch each other, operations such as dilations can lead to unwanted results.
Morphological filters on grey level images
Min and max operations can be applied to grey level images. Applications are for example contrast enhancement, edge detection, feature description, or pre-processing for segmentation.