Convolutional filters

Prerequisites

Before starting this lesson, you should be familiar with:

Learning Objectives

After completing this lesson, learners should be able to:

  • Understand the basic principle of a convolutional filter.

  • Apply convolutional filters to an image.

Motivation

Convolutional filters are very important, because

Concept map

graph TD SM("Small matrix of numbers\n(Kernel)") --> LM("Local, sliding multiplication") I("Image") --> LM LM --> FI("Filtered image")



Figure


Convolutional filtering example, using a 3x3 vertical edge detection filter.






Activities

Laplacian filter

The Laplacian edge filter (REF) is a very basic “edge enhancement” filter.

Observe that this filter mostly enhances spots, then ridges, then edges, while areas of equal intensity will be surpressed.


Show activity for:  

ImageJ Macro

// IJ-Macro that demonstrates the effects of a sipmle 
// Laplacian convolutional filter on an example image

// Open an input image
open("https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit__laplacian_test_input.tif");
rename("input");

// Pick a few (three) interesting locations in the image
// and manually compute the response to the
// Laplacian Kernel: 
// [0 -1 0]
// [-1 4 -1]
// [0 -1 0]
// Note down the results to compare with the below
// automated analysis.

// Duplicate the input image 
// such that the raw data is not overwritten
run("Duplicate...", "title=laplacian");

// Create and apply a simple Laplacian filter
run("Convolve...", "text1=[0 -1 0\n-1 4 -1\n0 -1 0\n]");

// Examine at the pixel intensities of the convolved image
// and study how the "response" of the image to the convolution 
// depends on the local structure of the input image.

skimage napari

# %%
# Instantiate napari
import napari
viewer = napari.Viewer()

# %%
# Create a Laplacian kernel
from scipy.ndimage import convolve
import numpy as np
laplacian_kernel = np.array([[0,-1,0],[-1,4,-1],[0,-1,0]])
print(laplacian_kernel)

# %%
# Open a test input image 
from OpenIJTIFF import open_ij_tiff
image, *_ = open_ij_tiff('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit__laplacian_test_input.tif')
viewer.add_image(image)

# %%
# Pick a few (three) interesting locations in the image
# and manually compute the response to the
# Laplacian Kernel.
# Note down the results to compare with the below
# automated analysis.

# %%
# Convolve the test image with the Laplacian kernel
# Try to exactly understand the output for every pixel
from scipy.ndimage import convolve
laplacian_image = convolve(image, laplacian_kernel)
viewer.add_image(laplacian_image)

# %% 
# Close the viewer (CI test requires this)
viewer.close()



Gaussian filter

Gaussian blur (REF) is the most commonly used image smoothing filter. In this activity you will learn how to apply a Gaussian blur filter to an image.


Show activity for:  

ImageJ Macro

// IJ-Macro that demonstrates the effects of a Gaussian 
// blur on a delta peak and example images

skimage napari

import numpy as np
from skimage.filters import gaussian
from scipy.ndimage import convolve
import matplotlib.pyplot as plt

# %%
# Create a delta peak image 
size = 5
image = np.zeros((size, size))
image[size // 2, size // 2] = 1
print(image)

# %%
# Apply a Gaussian blur with sigma 1 to the delta peak image
# in order to see the values of the Gaussian convolution kernel
sigma = 1
kernel = gaussian(image, sigma=sigma)
print(np.round(kernel, 2))

# %%
# Instantiate napari
import napari
viewer = napari.Viewer()

# %%
# Load an example image
# Observe that it is quite noisy
from OpenIJTIFF import open_ij_tiff
image, *_ = open_ij_tiff('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit__spots_local_background_with_noise.tif')
viewer.add_image(image)

# %%
# Use the Gaussian kernel to convolve the image
# - Observe that the smoothing did not really work for this image
# - Observe that the output of the convolve function has differnt datatype than the input
gaussian_image = convolve(image, kernel)
viewer.add_image(gaussian_image)
print(image.dtype, gaussian_image.dtype)  

# %%
# Use a Gaussian kernel with a larger sigma 
# to better smooth the image.
# This time, for simplicity, 
# we directly use the gaussian function.
gaussian_sigma7_image = gaussian(image, sigma=7)
viewer.add_image(gaussian_sigma7_image)

# %%
# Learning opportunities
# - Inspect the values of the gaussian kernel with sigma 7,
#   using again the trick to apply it to a delta peak image 

# %% 
# Close the viewer (CI test requires this)
viewer.close()
plt.close('all')



Laplacian of Gaussian filter

The Laplacian of Gaussian filter (REF) is a relatively advanced convolutional filter that enhances edges without creating too much noise.

In this activity you will learn how to create and apply a Laplacian of Gaussian filter to an image.


Show activity for:  

ImageJ Macro

open("/Users/tischer/Documents/training-resources/image_data/xy_8bit__spots_local_background_with_noise.tif");
rename("input");

// Compuate Laplacian of the image to enhance spots and edges
// Observe that this yields a very noisy resulting image
selectWindow("input");
run("Duplicate...", "title=Laplacian");
run("32-bit");
run("Convolve...", "text1=[0 -1 0\n-1 4 -1\n0 -1 0\n\n]");
run("Enhance Contrast", "saturated=0.35");

// To reduce the noise first apply a Gaussian 
// and then a Laplacian.
// This is a very common operation, also called 
// "Laplacian of Gaussian (LoG)".
// Visually inspect the image to determinte
// a sigma for the Gaussian kernel that 
// matches the size of the structures 
// that you want to enhance.
selectWindow("input");
run("Duplicate...", "title=LoG");
run("32-bit");
run("Gaussian Blur...", "sigma=7");
run("Convolve...", "text1=[0 -1 0\n-1 4 -1\n0 -1 0\n\n]");
run("Enhance Contrast", "saturated=0.35");


// Learning opportunities:
// - Explore how different values for the Gaussian sigma change the result
// - Observe how the results look very bad if you don't convert to float (32bit)

skimage napari

# %%
# This script demonstrates how to apply a Laplacian of Gaussian
# edge enhancing filter to an image

import napari
from OpenIJTIFF import open_ij_tiff
from skimage.filters import laplace, gaussian

# %%
# Instantiate napari
viewer = napari.Viewer()

# %%
# Load an example image
# Observe that it is quite noisy
image, *_ = open_ij_tiff('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xy_8bit__spots_local_background_with_noise.tif')
viewer.add_image(image)

# %%
# Apply a Laplacian to enhance spots and edges
# Observe that the resulting image is very noisy
laplacian_image = laplace(image)
viewer.add_image(laplacian_image)

# %%
# To reduce the noise first apply a Gaussian 
# and then the Laplacian.
# This is a very common operation, also called 
# "Laplacian of Gaussian (LoG)"
# Visually inspect the image to determinte
# a sigma for the Gaussian kernel that 
# matches the size of the structures 
# that you want to enhance.
gaussian_sigma = 7 
log_image = laplace(gaussian(image, gaussian_sigma))
viewer.add_image(log_image)

# %%
# Learning opportunities:
# - Explore how different values for the gaussian_sigma change the result

# %% 
# Close the viewer (CI test requires this)
viewer.close()






Assessment

Fill in the blanks

  1. What are the entries of a convolutional kernel that computes a local sum?
  2. You want to enhance horizontal filamentous structures in an image, how would a convolutional kernel for this look like?
  3. The entries of a convolutional kernel implementing an edge detection typically sum up to ___.

Solution

  1. The entries are all 1.
  2. A simple kernel to enhance horizontal lines could be: [[-1,-1,-1],[2,2,2],[-1,-1,-1]]
  3. The entries sum up to zero, such that the result of the convolution in image regions without edges is zero.




Follow-up material

Recommended follow-up modules:

Learn more: