Morphological filters

Prerequisites

Before starting this lesson, you should be familiar with:

Learning Objectives

After completing this lesson, learners should be able to:
  • Understand how to design morphological filters using rank filters

  • Execute morphological filters on binary or label images and understand the output

Motivation

Morphological filters (MFs) are used to clean up segmentation masks and achieve a change in morphology and/or size of the objects. For example, MFs are used to remove wrongly assigned foreground pixels, separate touching objects, or identify objects boundaries.

Concept map

graph TD subgraph opening erode("Erode (min)") --> dilate("Dilate (max)") end subgraph closing dilate2("Dilate (max)") --> erode2("Erode (min)") end subgraph rank operations any("...") end BI("Binary/label image") --> SE("structuring element") SE .-> erode SE .-> dilate2 SE .-> any dilate .-> BIM erode2 .-> BIM any .-> BIM("Modified binary/label image")



Figure


Upper row - Example of a workflow using morphological filters to improve the segmentation and compute the edge of the nuclei for further intensity measurements if necessary measurements have to be made on the periphery. Image on top left is a 2 channel intensity image (where channel 1 shows nuclear membrane and channel 2 shows dna staining). Lower row - Image level description of dilation and erosion operation using a 3x3 structuring element (left side). Morphological filters applied in series, e.g. opening (i.e. erosion followed by dilation) and closing (i.e. dilation followed by erosion), can achieve very useful results (right side). Red and green arrows show erosion and dilation respectively



Activities

Dilation and erosion of binary


Show activity for:  

ImageJ Macro

run("Close All");
//Make sure black background in Process > Binary > Options is set to true
setOption("black background", true);

open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__three_spots_different_size.tif");
// Image > Rename
rename("binary");
run("Set... ", "zoom=400 x=29 y=25");
// Erosion, use default binary IJ binary operations
// It is a radius 1 squared SE, i.e. 3x3 SE

// Image > Duplicate, name it eroded
run("Duplicate...", "title=eroded");
//Process > Binary > Erode
run("Erode");
run("Set... ", "zoom=400 x=29 y=25");
// Apply second erosion will remove small dot
// Image > Duplicate, name it eroded_twice
run("Duplicate...", "title=eroded_twice");
// Process > Binary > Erode
run("Erode");
run("Set... ", "zoom=400 x=29 y=25");
// Use MorpholibJ and a radius 2, i.e. 5x5 squared structuring element
// This corresponds to 2 sequntial 3x3 erosions
selectWindow("binary");
// Plugins > MorpholibJ > Filtering > Morphological Filters 
// Select Erosion, Element Square 
run("Morphological Filters", "operation=Erosion element=Square radius=2");
rename("erosion_radius2");
run("Set... ", "zoom=400 x=29 y=25");
// Dilation, use MorpholibJ
selectWindow("binary");
// Plugins > MorpholibJ > Filtering > Morphological Filters
// Use MorpholibJ and a radius 2, i.e. 5x5 squared structuring element
run("Morphological Filters", "operation=Dilation element=Square radius=1");
rename("dilation_radius1");
run("Set... ", "zoom=400 x=29 y=25");
// Dilation, use MorpholibJ
selectWindow("binary");
// Plugins > MorpholibJ > Filtering > Morphological Filters
// Use MorpholibJ and a radius 2, i.e. 5x5 squared structuring element
run("Morphological Filters", "operation=Dilation element=Square radius=2");
rename("dilation_radius2");
run("Set... ", "zoom=400 x=29 y=25");
run("Tile")

skimage napari

# %% 
# Dilation and erosion of a binary image

# %%
# Import python packages.
from OpenIJTIFF import open_ij_tiff
from napari.viewer import Viewer
from skimage.morphology import square, disk
from skimage.morphology import erosion, dilation
import numpy as np

# %%
# Create a napari_viewer 
napari_viewer = Viewer()

# %%
# Open and view image
image, *_ = open_ij_tiff("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__three_spots_different_size.tif")
napari_viewer.add_image(image)

# %%
# Check datatype and pixel values 
# - Ensure that this is a binary image
# - Check how the binary values are encoded (could be in principle: true, false, 0, 1, 255)
print(image.dtype)
print(np.unique(image))

# %%
# Perform [erosion](https://scikit-image.org/docs/stable/api/skimage.morphology.html#skimage.morphology.erosion) and [dilation](https://scikit-image.org/docs/stable/api/skimage.morphology.html#skimage.morphology.dilation) using default setting
#

# %%
# Perform erosion and dilation with a cross-shaped / disk(1) structural element
# This element has connectivity = 1
#
print(disk(1))
eroded = erosion(image, footprint = disk(1))
dilated = dilation(image, footprint = disk(1))

# Add images to napari and observe:
# - The single pixel disappeared with erosion
# - The single pixel became a cross with dilation. This is in fact the form of the structuring element
# - For the dilation no pixels have been added (diagonally) at corners, because the disk(1) has only horizontal and vertical "1" connectivity
napari_viewer.add_labels(eroded)
napari_viewer.add_labels(dilated)

# %%
# Now try with a structuring element with connectivity 2 (3x3 square).
print(square(3))
eroded_square3 = erosion(image, footprint = square(3))
dilated_square3 = dilation(image, footprint = square(3))

# %%
# View images in napari
#
napari_viewer.add_labels(eroded_square3)
napari_viewer.add_labels(dilated_square3)

# %%
# Learning opportunity:
# Try with a bigger square (e.g. square(5))
# or a different structuring element (e.g. disk(1))
# Also refer to https://scikit-image.org/docs/stable/api/skimage.morphology.html



Opening and closing of binary


Show activity for:  

ImageJ Macro

run("Close All");
//Make sure black background in Process > Binary > Options is set to true
setOption("black background", true);
open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__for_open_and_close.tif")
rename("binary");
run("Set... ", "zoom=400 x=29 y=25");
// Opening, use default binary IJ binary operations in sequence
run("Duplicate...", "title=opening");
//Process > Binary > Erode
run("Erode");
//Process > Binary > Dilate
run("Dilate");
run("Set... ", "zoom=400 x=29 y=25");
// See how the thin structure disappear

// Opening, use default binary IJ binary operations
selectWindow("binary");
run("Duplicate...", "title=opening2");
//Process > Binary > Open
run("Open");
run("Set... ", "zoom=400 x=29 y=25");

// Opening, use MorpholibJ, try different radii
selectWindow("binary");
// Plugins > MorpholibJ > Filtering > Morphological Filters
run("Morphological Filters", "operation=Opening element=Square radius=1");
rename("binary-Opening_radius=1");
run("Set... ", "zoom=400 x=29 y=25");
selectWindow("binary");
// Plugins > MorpholibJ > Filtering > Morphological Filters
run("Morphological Filters", "operation=Opening element=Square radius=3");
rename("binary-Opening_radius=3");
run("Set... ", "zoom=400 x=29 y=25");
// see how also the small blob disappear, side of large blob are deformed


// Closing, use MorpholibJ
selectWindow("binary");
// Plugins > MorpholibJ > Filtering > Morphological Filters
run("Morphological Filters", "operation=Closing element=Square radius=1");
rename("binary-Closing_radius=1");
run("Set... ", "zoom=400 x=29 y=25");
// Fill the hole in big blob
// Closes the gap
run("Tile")

skimage napari

# %% [markdown]
# ### Opening and closing of binary
#
# Part of teaching module [Morphological filtering](https://neubias.github.io/training-resources/filter_morphological/index.html#openclose)

# %%
# Import python packages.
from OpenIJTIFF import open_ij_tiff
from napari.viewer import Viewer
from skimage.morphology import square, disk
from skimage.morphology import erosion, dilation
from skimage.morphology import opening, closing

# %%
# Explore opening and closing combining erosion and dilation.
# Verify that they give the same results.
fpath = "https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__for_open_and_close.tif"
image, _, _, _ = open_ij_tiff(fpath)
napari_viewer = Viewer()
napari_viewer.add_image(image)

# %% [markdown]
# Morphological *opening* is the *dilation* of an *eroded* image using the same structuring element

# %%
# Opening operation
eroded = erosion(image, footprint = square(3))
opened = dilation(eroded, footprint = square(3))
napari_viewer.add_labels(eroded)
napari_viewer.add_labels(opened)

# %% [markdown]
# Appreciate how the *opening* operation removed thin structures (< structuring element size) and smoothen objects

# %%
# Opening operations are so common that often they have their own command
opened_1step = opening(image, square(3))
print((opened==opened_1step).all())


# %% [markdown]
# Morphological *closing* is the *erosion* of a *dilated* image using the same structuring element

# %%
# Closing: fill holes, connect gaps.
dilated = dilation(image, footprint = square(3))
closed = erosion(dilated, square(3))
napari_viewer.add_labels(dilated)
napari_viewer.add_labels(closed)

# %%
# Closing operations are so common that often they have their own command
closed_1step = closing(image, square(3))
print((closed==closed_1step).all())

# %% [markdown]
# Appreciate how the *closing* operation filled holes and connected gaps (< structuring element size)

# %% [markdown]
# **Learning opportunity**\
# Try using the default structuring element (`disk(1)`)\
# Explain what has changed and why. Hint: Look at the dilated image prior to erosion



Morphological internal gradient of binary


Show activity for:  

ImageJ Macro

run("Close All");
//Make sure black background in Process > Binary > Options is set to true
setOption("black background", true);

open("https://raw.githubusercontent.com/NEUBIAS/training-resources/master/image_data/xy_8bit_binary__h2b.tif");
rename("binary");

// Internal gradient is the original - eroded image
// Shown as step by step process
// Erode image
// Plugins > MorpholibJ > Filtering >  Morphological Filters
//   Operation = Erosion
//   Element = Square
//   Radius (in pixels) = 1
run("Morphological Filters", "operation=Erosion element=Square radius=1");
rename("eroded")
// Process > Image Calculator ... 
//    image1 = binary
//    operation = subtract
//    image2 = binary-Erosion
//    [x] Create enew window
imageCalculator("Subtract create", "binary","eroded");
rename("internal_gradient");

// Add internal_gradient as overlay
selectImage("binary");
// Image > Overlay > Add..
run("Add Image...", "image=internal_gradient x=0 y=0 opacity=50");

//Internal gradient with MorpholibJ

selectWindow("binary");
// Plugins > MorpholibJ > Filtering >  Morphological Filters
//   Operation = Erosion
//   Element = Square
//   Radius (in pixels) = 1
run("Morphological Filters", "operation=[Internal Gradient]  element=Square radius=1");

run("Tile")

skimage napari

# %% 
# Morphological internal gradient of a binary image

# %%
from OpenIJTIFF import open_ij_tiff
from napari.viewer import Viewer
from skimage.morphology import square, disk
from skimage.morphology import erosion, dilation

# Create a napari_viewer and visualize image and labels.
napari_viewer = Viewer()

# %%
# Explore internal gradient.
fpath = "https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit_binary__h2b.tif"
image, _, _, _ = open_ij_tiff(fpath)

# Internal gradient is the difference between the image and the eroded version of it.
eroded = erosion(image)
internal_gradient = image - eroded

# %%
# Create a napari_viewer and visualize images.
napari_viewer.add_image(image)
napari_viewer.add_labels(eroded)
napari_viewer.add_labels(internal_gradient)

# %% [markdown]
# The internal gradient represents the inner edge of the object.\
# Discuss when and how this can be useful.

# %% [markdown]
# **Learning opportunity**
# * Compute the external gradient (dilation - image)
# * Try different sized structuring elements for the dilation
# * What controls the thickness of the edge?
# * Compute the central gradient (dilation - erosion)



Workflow measure intensity on the nuclear membrane

In image xyc_16bit__nup__nuclei.tif we would like to measure the intensity along the nuclear membrane (channel 1) using the information from the DNA (channel 2) using following workflow:


Show activity for:  

ImageJ GUI

// TODO adapt to new text

  • Open the image xy_8bit_binary__nuclei_noisy.tif
  • [Plugins > MorpholibJ > Filtering > Morphological Filters]
    • operation : Opening
    • element : Square
    • radius : 1
  • [Plugins > MorpholibJ > Filtering > Morphological Filters]
    • operation : Closing
    • element : Square
    • radius : 16
  • An alternative to the closing operation is also [Plugins > MorpholibJ > Filtering > Fill Holes (Binary/Gray)]

  • Open xy_8bit_labels__nuclei.tif
  • [Plugins > MorpholibJ > Filtering > Morphological Filters]
    • Operation : Internal Gradient
    • Element : Square
    • Radius : 3
  • [ Image › Rename… ]
    • “rim”
  • Open xyc_16bit__nup_nuclei
  • [Image > Duplicate…]
    • Title : Ch1
    • Duplicate hyperstack
    • Channels (c): 1
  • [Plugins > MorpholibJ > Analyze > Intensity Measurements 2D/3D]
    • Input : Ch1
    • Labels : rim

ImageJ Macro

// TODO Adapt to new text !
run("Close All");
open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyc_16bit__nup_nuclei/xy_8bit_binary__nuclei_noisy.tif");
run("Morphological Filters", "operation=Opening element=Square radius=1");
run("Morphological Filters", "operation=Closing element=Square radius=16");
run("Connected Components Labeling", "connectivity=4 type=[8 bits]");
run("glasbey_on_dark");

run("Close All");
open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyc_16bit__nup_nuclei/xy_8bit_labels__nuclei.tif");
run("Morphological Filters", "operation=[Internal Gradient] element=Square radius=3");
rename("rim");
open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyc_16bit__nup_nuclei.tif");
run("Duplicate...", "title=Ch1 duplicate channels=1");
run("Intensity Measurements 2D/3D", "input=Ch1 labels=rim mean stddev max min median  numberofvoxels");



Create a cytoplasmic ring label mask


Show activity for:  

skimage napari

# %% 
# Create a cytoplasmic ring of a nuclei label mask

# %%
from OpenIJTIFF import open_ij_tiff
import napari
from skimage.morphology import square, disk
from skimage.morphology import dilation
from skimage.segmentation import expand_labels

# Create a napari_viewer and visualize image and labels.
viewer = napari.viewer.Viewer()

# %%
# open the label mask image
labels, *_ = open_ij_tiff("https://github.com/NEUBIAS/training-resources/raw/master/image_data/watershed/xy_8bit_labels__nuclei.tif")
viewer.add_labels(labels)

# %%
# dilate the label image and inspect the result in napari
# observe that the nuclei with the larger label index grow into ones with smaller label indices
# the reason for this is that the implementation of the dilation is a local maximum filter
# this is not a useful result for creating cytoplasmic rings
dilated_labels = dilation(labels, disk(10))
viewer.add_labels(dilated_labels)

# %%
# now dilate the label image with the special "expand_labels" command 
# observe that now the nuclei do not grow into each other
expanded_labels = expand_labels(labels, 10)
viewer.add_labels(expanded_labels)

# %%
# create a cytoplasmic ring by subtracting two differently dilated 
# nuclei label masks from each other
# note that here we subtract slightly dilated nuclei to leave some gap 
# between the nucleus and the cytoplasmic ring
ring_labels = expand_labels(labels, 10) - expand_labels(labels, 2) 
viewer.add_labels(ring_labels)

# %%
# prevent napari from quitting when exectuing from a scripting environment
# napari.run()






Assessment

Fill in the blanks

Using those words fill in the blanks: closing, opening, min, shrinks, decreases, enlarges, max.

  1. An erosion _____ objects in a binary image.
  2. An erosion in a binary image _____ the number of foreground pixels.
  3. A dilation _____ objects in a binary image.
  4. An erosion of a binary image corresponds to a ___ rank operation.
  5. An dilation of a binary image corresponds to a ___ rank operation.
  6. A dilation followed by an erosion is called ___.
  7. An erosion followed by a dilation is called ___ .

Solution

  1. shrinks
  2. decreases
  3. enlarges
  4. min
  5. max
  6. closing
  7. opening

True of false?

Discuss with your neighbour!

  1. Morphological openings on binary images never decrease the number of foreground pixels.
  2. Morphological closings on binary images never decreases the number of foreground pixels.
  3. Performing a morphological closing twice in a row does not make sense, because the second closing does not further change the image.
  4. Performing a morphological closing with radius 2 element is equivalent to two subsequent closing operation with radius 1.

Solution

  1. False
  2. True
  3. True
  4. False

Explanations

Rank filters

In the region defined by the structuring element, pixel elements are ranked/sorted according to their values. The pixel in the filtered image is replaced with the corresponding sorted pixel (smallest = min, greatest = max, median ). See also Median filter. Morphological filters corresponds to one or several rank filters applied to an image.

Morphological filters on binary images

A typical application of these filters is to refine segmentation results. A max-filter is called dilation whereas a min-filter is called erosion. Often rank filters are applied in a sequence. We refer to a closing operation as a max-filter followed by a min-filter of the same size. An opening operation is the inverse, a min-filter followed by a max-filter.

Opening operations will:

  • Remove small/thin objects which extent is below the size of the structuring element
  • Smooth border of an object

Closing operations:

  • Fill small holes below the size of the structuring element
  • Can connect gaps

Image subtraction using eroded/dilated images allows to identify the boundary of objects and is referred to morphological gradients:

  • Internal gradient: original - eroded
  • External gradient: dilated - original
  • (Symmetric) gradient: dilated - eroded

Fill holes operation is a slightly more complex morphological operation. It is used to identify background pixels surrounded by foreground pixels and change their value to foreground. Algorithmically there are several ways to achieve this.

Morphological filters on label images

Morphological filters work also on label images. If the objects are not touching this will achieve the expected result for each label. However, when objects touch each other, operations such as dilations can lead to unwanted results.

Morphological filters on grey level images

Min and max operations can be applied to grey level images. Applications are for example contrast enhancement, edge detection, feature description, or pre-processing for segmentation.




Follow-up material

Recommended follow-up modules:

Learn more: