After completing this lesson, learners should be able to:
Understand the basic principle of a neighborhood filter.
Apply basic neighborhood filters to an image.
Motivation
Images are quite often noisy or have other issues that make them hard to segment, e.g. by means of a simple intensity threshold. Neighborhood filters are often used to enhance the images in order to facilitate better performance of segmentation algorithms.
Image filtering with a pixel neighborhood. (a) Raw intensity image with pixel neighborhood (structuring element (SE), outer green box) and central pixel (inner orange box) on which the filtering operation will be performed. (b) Pixel values in the neighborhood. (c) X is the value that would be replaced after operation (indicated as op). Here, max, mean and variance operations are used. Note - One has to carefully look at the data type of the image as some operations can produce large/floating point values. (d) Different SEs (neighborhood in green and affected pixel in orange) top left - SE completely inside image boundaries; top right - SE at image boundaries (padding needed); bottom left - SE with different shape; bottom right - Line SE.
It is helpful to first duplicate the image: [Image > Duplicate…] or [Ctrl-Shift-D]
Title = mean_1
Apply a mean filter to the image such that you can binarize the image into exactly three disjoint foreground regions (the three nuclei). The resulting mask should look like xy_8bit_binary__nuclei_very_noisy.tif
[Process > Filters > Mean…]
Radius = 1 pixels
[Image > Adjust > Threshold…]
Dark Background
Lower threshold level = 31
Higher threshold level = 255
Press Set
Press Apply
[Image > Lookup Tables > Invert LUT]
Repeat the procedure with radii = 2, 5, 7
What is the smallest size for the mean filter that does the job?
Radius = 2 pixels and Lower threshold level = 30 (other values can also be chosen) can remove the noise and find the appropriate nuclei size after manual thresholding
# %% [markdown]
# ## Applying mean filters to an image
# %%
# Instantiate the napari viewer
importnaparifromOpenIJTIFFimportopen_ij_tiffviewer=napari.Viewer()# %%
# Read the intensity image
image,axes,scales,units=open_ij_tiff('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit__nuclei_very_noisy.tif')# %%
# View the intensity image
viewer.add_image(image)# %%
# Appreciate that one cannot segment the nuclei by a simple intensity threshold
binary_image=image>40viewer.add_image(binary_image)# %%
# Define a circular structural element with a radius of 1 pixel
fromskimage.morphologyimportdiskradius=1disk_radius1=disk(radius)print(disk_radius1)# %%
# Apply a mean filter with the structural element
fromskimage.filters.rankimportmeanmean_disk_radius1=mean(image,disk_radius1)# Add the filtered image to napari
# Napari: Zoom in to appreciate that the filtered image
# contains the local mean values of the raw image
viewer.add_image(mean_disk_radius1)# %%
# One still cannot readily segment the nuclei
# (rerun the below code with different thresholds)
binary_mean_disk_radius1=mean_disk_radius1>35viewer.add_image(binary_mean_disk_radius1)# %%
# Apply mean filter with a disk of radius 3
mean_disk_radius3=mean(image,disk(3))viewer.add_image(mean_disk_radius3)# %%
# Now the nuclei can be segmented
binary_mean_disk_radius3=mean_disk_radius3>32viewer.add_image(binary_mean_disk_radius3)
Apply a variance filter [ Process > Filter > Variance… ] to segment the cell regions from the background
Hints:
Convert image to float, because the filter may yield high values [ Image > Type > 32-bit ]
What filter radius and threshold yield a good segmentation?
Solution
[ Image › Rename…]
Title = input
[ Image › Duplicate… ]
Title = variance_5
[ Image > Type > 32-bit ]
In variance calculation, pixel values can exceed 255 which is the maximum value that can be achieved in current bit depth (unsigned 8-bit) of input image.
[ Process › Filters › Variance… ]
Radius = 15 pixels
[ Image › Adjust › Threshold… ]
([x] Dark Background)
Lower threshold level = 1.5
Higher threshold level = 1e30
Press Set
Press Apply
Press Convert to Mask
ImageJ Macro
// File > Close All
run(“Close All”);
// File > Open…
open(“C:/Users/akhan/training-resources/image_data/xy_8bit__em_mitochondria_er.tif”);
//Press Set button (example upper and lower threshold values are (113, 255))
setThreshold(113, 255);
// Press Apply button
run(“Convert to Mask”);
// Image › Lookup Tables › InvertLUT… (if highest values are darker and lowest brighter)
run(“Invert LUT”);
skimage napari
# %% [markdown]
# ## Applying variance filters to an image
# %%
# Instantiate the napari viewer
importnaparifromOpenIJTIFFimportopen_ij_tiffviewer=napari.Viewer()# %%
# Read the intensity image and add it to napari
image,axes,scales,units=open_ij_tiff('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit__em_mitochondria_er.tif')viewer.add_image(image)# %%
# Appreciate that one cannot readily segment the sample by a simple intensity threshold
binary_image=image<230viewer.add_image(binary_image)# %%
# Apply a variance filter
# Convert to uint16 to be able to accommodate
# the square of the unit8 image
importnumpyasnpprint(image.dtype)uint16_image=image.astype(np.uint16)# Use ndimage instead of skimage,
# because skimage.filters.rank.mean
# only works up to unit16, which here specifically
# would have been sufficient. However if the input image
# would have been uint16 we would have needed unit32
# for the variance.
fromscipyimportndimagesqr_mean_image=ndimage.uniform_filter(uint16_image,(11,11))**2mean_sqr_mean=ndimage.uniform_filter(uint16_image**2,(11,11))var_image=mean_sqr_mean-sqr_mean_imageviewer.add_image(var_image)# %%
# Segment the foreground by applying a threshold
# (Note that in this specific case a simple mean filter might have been sufficient
# to achieve the same result, because the sample is on average darker than the background)
binary_var_image=var_image>100viewer.add_image(binary_var_image)
Assessment
Fill in the blanks
Fill in the blanks, using these words: decrease, increase, size, structuring element, large
A synonym for neighborhood is __
The filter radius characterize the filter ___
___ filter size can cause a loss of details in the filtered image
Filter can be used to __ the noise in an image
The usage of filters can __ the quality of image segmentation/binarization
Solution
A synonym for neighborhood is structuring element (SE)
The filter radius characterize the filter size
large filter size can cause a loss of details in the filtered image
Filter can be used to decrease the noise in an image
The usage of filters can increase the quality of image segmentation/binarization
Explanations
Neighborhood filters comprise two ingredients: a definition of the pixel neighborhood (size and shape) and a mathematical recipe what to compute on this neighborhood.
The result of this computation will be used to replace the value of the central pixel in the neighborhood. This procedure can be applied to several (all) pixels of an image to obtain a filtered image. The animation shows a square neighborhood (3x3) applied to the inner pixels of the image.
There are tons of different neighborhood filters, and you can also invent one!
The neighborhoods
The neighborhood of a pixel is also called a structuring element (SE) and can have various sizes and shapes.
Here, we use one of the simplest and most widely used neighborhoods, namely a circular neighborhood, which is defined by a certain radius. We will explore other shapes of SEs in more detail in a dedicated module.
Padding
Since the filtering operation takes into account all the directions of extent of SE, border pixels would be affected in a different way and one has to decide that which values they should assume. Padding is the operation of adding an additional layer around your data to get more accurate values after filtering process. It also gives you option to retain same dimensions for your data after filtering. Common padding methods are using zeros or to mirror/replicate the border pixel values.
The math
There are many ways how to cleverly compute on a pixel neighborhood. For example, one class of computations is called convolutional filters, another is called rank filters. Here, we focus on the relatively simple mean and variance filters.
Best practice
As usual, everything depends one the scientific question, but maybe one could say to use a filter that changes the image as little as possible.