Projections

Prerequisites

Before starting this lesson, you should be familiar with:

Learning Objectives

After completing this lesson, learners should be able to:
  • Project multi-dimensional image data into lower dimensions

  • Understand the differences between projection modes such as max, sum, and mean

Motivation

Viewing image data that has more than two dimensions is difficult, because computer monitors are 2-D. Thus, it often is very useful to project the data into a 2-D representation. Of course, doing such a projection will loose information. Thus, performing projections without compromising the scientific integrity of the data is not easy and should only be done with a sufficient understanding of the various methods.

Concept map

graph TD ND("N dimensional image") --> PM("Simple projection") PM -->|has| A("Axis") PM -->|has| M("Method: max, sum, ...") PM -->|creates| LD("N-1 dimensional image")



Figure




Activities

Explore max and sum projections


Show activity for:  

ImageJ GUI Reslice

Open an image

  • Open xyz_16bit__spots.tif
  • Drag and drop above link onto Fiji or [File > Import > URL…]: https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_16bit__spots.tif

Z axis projection

  • run("Z Project...", "projection=[Max Intensity]");
  • run("Z Project...", "projection=[Sum Slices]");
  • Compare pixel values and image data types
  • Appreciate that the data type of the sum projection is different to the original image.
  • Discuss whether the data type change was necessary in this specific case.

X and Y axis projection

There is no easy way to project along the x or y axis in ImageJ. We need to rearrange the stack such that the new z axis is the one along which to project.

  • Maximum projection along x axis
    • run("Reslice [/]...", "start=Left avoid");
      • Left/Right/Top/Bottom = as if looking from the left(east)/right(west)/top(north)/bottom(south) onto the stack on your screen (Top does not mean to look on the stack from the top as in along the z-direction).
      • [X] avoid interpolation (otherwise it will create new pixels by interpolation)
        • The output spacing argument will be ignored if we do not interpolate.
    • Note that the image has the same number of pixels, but rearranged.
    • Note that also the voxel sizes run("Properties..) have changed.
    • run("Z Project...", "projection=[Max Intensity]");
  • Maximum projection along y axis
    • run("Reslice [/]...", "start=Top avoid");
    • run("Z Project...", "projection=[Max Intensity]");

Appearance of anisotropic images

Notice that the images do not look correct in a physical sense, but squashed. This is due to a mismatch of the data (voxel) space and physical space.

  • Select one of the projected (x or y axis) images
  • run("Properties...")
  • Observe that the voxel sizes are correct, but the ImageJ viewer does not take them into account for rendering.
  • Opening the same image in BigDataViewer paints a different picture: [ Plugins > BigDataViewer > Open Current Image…]

Resampling (optional)

To achieve a more correct appearance in physical space in the ImageJ viewer we need to up-scale the image and add more voxels. Note that while this is good for visualization, it does change the data and should thus be done with care.

  • Compute the scaling factor: 0.4 / 0.0941345 = 4.249239
  • Rescale the x-projection: run("Scale...", "x=1.0 y=4.249239 z=1.0 interpolation=None average create");
  • Rescale the y-projection: run("Scale...", "x=4.249239 y=1.0 z=1.0 interpolation=None average create");

ImageJ GUI CLIJ2

Install CLIJ2

In ImageJ one can project along the z axis (e.g. [Image > Stacks > Z Project …]), but there is no easy way to project along the x or y axis. Thus let’s install the very useful update site: CLIJ2.

  • [Help > Update…]
  • [Manage update sites]
  • clij
  • clij2

Open example image

  • Open xyz_16bit__spots.tif.
  • Drag and drop above link onto Fiji or [File > Import > URL…]: https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_16bit__spots.tif

Z axis projection

  • Sum projection: [Plugins › ImageJ on GPU (CLIJ2) › Projections › Sum-Z-projection on GPU]
  • Maximum projection: [Plugins › ImageJ on GPU (CLIJ2) › Projections › Max-Z-projection on GPU]
  • Compare the pixel values and data types of the two images.
  • Appreciate that the data type of the sum projection is different from that of the original image.
  • Discuss whether the data type change was necessary in this specific case.

X and Y axis projection

  • Maximum projection x: [Plugins › ImageJ on GPU (CLIJ2) › Projections › Max-X-projection on GPU]
  • Maximum projection y: [Plugins › ImageJ on GPU (CLIJ2) › Projections › Max-Y-projection on GPU]

Appearance of anisotropic images

Notice that the x and y projected images do not look correct in a physical sense, but squashed. This is due to a mismatch of the data (voxel) space and physical space.

  • Select one of the projected (x or y axis) images
  • run("Properties...")
  • Observe that the voxel metadata is not maintained by CLIJ2
  • For y projection change the image properties to width = 0.0941345 and height = 0.4
  • For x projection change the image properties to width = 0.4 and height = 0.0941345
  • Observe that even with correct voxel size the ImageJ viewer does not take them into account for rendering.
  • Opening the same image in BigDataViewer paints a different picture: [ Plugins > BigDataViewer > Open Current Image…]

Resampling (optional)

To achieve a more correct appearance in physical space in the ImageJ viewer we need to up-scale the image and add more voxels. Note that while this is good for visualization, it does change the data and should thus be done with care.

  • Compute the scaling factor: 0.4 / 0.0941345 = 4.249239
  • Rescale the x-projection: run("Scale...", "x=1.0 y=4.249239 z=1.0 interpolation=None average create");
  • Rescale the y-projection: run("Scale...", "x=4.249239 y=1.0 z=1.0 interpolation=None average create");

ImageJ GUI TransformJ

Open an image

  • Open xyz_16bit__spots.tif
  • Drag and drop above link onto Fiji or [File > Import > URL…]: https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_16bit__spots.tif

Z axis projection

  • run("Z Project...", "projection=[Max Intensity]");
  • run("Z Project...", "projection=[Sum Slices]");
  • Compare pixel values and image data types
  • Appreciate that the data type of the sum projection is different to the original image.
  • Discuss whether the data type change was necessary in this specific case.

X and Y axis projection

There is no easy way to project along the x or y axis in ImageJ. We need to rotate the stack such that the new z-axis is the one along which to project. For rotating we use TransformJ, which has many useful functions for dealing with 3D data. This requires the very useful update site: ImageScience.

  • Install TransformJ: [ Help > Update… ]: [Manage Update Site]: [X] ImageScience
  • run("TransformJ Turn", "z-angle=0 y-angle=90 x-angle=0");
  • run("Z Project...", "projection=[Max Intensity]");
  • run("TransformJ Turn", "z-angle=0 y-angle=0 x-angle=90");
  • run("Z Project...", "projection=[Max Intensity]");

Appearance of anisotropic images

Notice that the images do not look correct in a physical sense, but squashed. This is due to a mismatch of the data (voxel) space and physical space.

  • Select one of the projected (x or y axis) images
  • run("Properties...")
  • Observe that the voxel sizes are correct, but the ImageJ viewer does not take them into account for rendering.
  • Opening the same image in BigDataViewer paints a different picture: [ Plugins > BigDataViewer > Open Current Image…]

Resampling (optional)

To achieve a more correct appearance in physical space in the ImageJ viewer we need to up-scale the image and add more voxels. Note that while this is good for visualization, it does change the data and should thus be done with care.

  • Compute the scaling factor: 0.4 / 0.0941345 = 4.249239
  • Rescale the x-projection: run("Scale...", "x=1.0 y=4.249239 z=1.0 interpolation=None average create");
  • Rescale the y-projection: run("Scale...", "x=4.249239 y=1.0 z=1.0 interpolation=None average create");

skimage napari

# %% [markdown]
# ## Explore Max and Sum Projections
#
#

# %%
import sys
sys.path.append("C:\\Users\\akhan\\python_course")

# %%
# Import python packages.
from OpenIJTIFF import open_ij_tiff
import numpy as np
from napari.viewer import Viewer
from skimage.transform import rescale

# %%
# Instantiate the napari viewer
viewer = Viewer()

# %%
# Open a 3D image
image_url = 'https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_16bit__spots.tif'
image, axes, scales, units = open_ij_tiff(image_url)
viewer.add_image(image)

# %%
# Check image dimension
# Axis order is 0=z, 1=x, 2=y
image.shape



# %%
# Maximum projection along z-axis. i.e. axis = 0
max_z_image = np.max(image, axis=0)
viewer.add_image(max_z_image)

# %% [markdown]
# Use *Toggle grid mode (Ctrl + G)* in **Napari-GUI** to view images side by side

# %%
# Sum projection along z-axis
sum_z_image = np.sum(image, axis=0)


# %%
# Appreciate that changing the data type during sum projections is useful
print(np.iinfo(image.dtype)) # range of values that the image's datatype can represent

# %%
# Maximum value that could occur during sum projection 
# is number of slices (image.shape[0]) multiplied by the maximal possible value for each slice
max_possible_value = image.shape[0]*np.iinfo(image.dtype).max
print("Maximum possible pixel value during sum projection %d"  % max_possible_value)

# %%
# luckily numpy changed the data type during projection
print(np.iinfo(sum_z_image.dtype)) # range of values that the new data type can represent

# %%
# Display the image in Napari
viewer.add_image(sum_z_image)

# %%
# BUG - Napari has an issue displaying uint32. 
# Convert to float to properly visualize the image
sum_z_image_float = sum_z_image.astype(float)
viewer.add_image(sum_z_image_float) 

# %%
# Maximum projection along x-axis and y-axis
max_x_image = np.max(image, axis=2)
max_y_image = np.max(image, axis=1)
viewer.add_image(max_x_image)
viewer.add_image(max_y_image)

# %%
# Due to the anisotropic pixel size the x and y projections appear squashed
# We can rescale the image to make it appear physically correct
np.set_printoptions(precision=3)
print(f"Voxel size in micrometer [Z, Y, X]: {np.array(scales)}")

dz = scales[0]
dy = scales[1]
dx = scales[2]

# %%
# The axes of the max_y_image are 0=z,1=x
# The z axes is squashed thus we enlarge it to make the image isotropic
rescaled_max_y_image = rescale(max_y_image, [dz/dy,1])  
viewer.add_image(rescaled_max_y_image)

# %%
# The max_x_image could be rescaled in the same way
rescaled_max_x_image = rescale(max_x_image, [dz/dy,1])  
viewer.add_image(rescaled_max_x_image)


# %%
rescaled_max_x_image.shape


# %%
rescaled_max_y_image.shape

# %%



How image content affects sum projection


Show activity for:  

ImageJ GUI Reslice

Open example image

Z axis projection

  • Perform a z axis sum projection:
    • run("Z Project...", "projection=[Sum Intensity]");
  • What is the highest pixel value in the z sum projection? (use e.g. [Analyze > Histogram])

Y axis projection

  • Perform a y axis sum projection:
    • run("Reslice [/]...", "start=Top avoid");
    • run("Z Project...", "projection=[Sum Intensity]");
  • What is the highest pixel value in the y sum projection?
  • You should find that the value for the y axis is higher, explain why this could be expected from the morphology of the golgi in the original image.
  • What would you expect for doing above exercise with maximum projections? Also two different values or two times the same value?

Answers

  • The highest value in the z sum projection is 94558.
  • The highest value in the y sum projection is 165401.
  • The Golgi is elongated along the y axis, thus a sum projection adds up a lot of high values along this axis.
  • For maximum projections you would expect the same value.

ImageJ GUI CLIJ2

Install CLIJ2

In ImageJ one can project along the z axis (e.g. [Image > Stacks > Z Project …]), but there is no easy way to project along the x or y axis. Thus let’s install the very useful update site: CLIJ2.

  • [Help > Update…]
  • [Manage update sites]
  • clij
  • clij2

Open example image

Z axis projection

  • Sum projection: [Plugins > ImageJ on GPU (CLIJ2) > Projections > Sum-Z-projection on GPU]
  • What is the highest pixel value in the z sum projection? (use e.g. [Analyze > Histogram])

Y axis projection

  • Sum projection: [Plugins > ImageJ on GPU (CLIJ2) > Projections > Max-Y-projeciton on GPU]
  • What is the highest pixel value in the y sum projection?
  • You should find that the value for the y axis is higher, explain why this could be expected from the morphology of the golgi in the original image.
  • What would you expect for doing above steps with maximum projections? Also two different values or two times the same value?

Answers

  • The highest value in the z sum projection is 94558.
  • The highest value in the y sum projection is 165401.
  • The Golgi is elongated along the y axis, thus a sum projection adds up a lot of high values along this axis.
  • For maximum projections you would expect the same value.






Assessment

Fill in the blanks

  1. A projection ___ the number of dimensions in an image.
  2. The pixel values in a sum projection will typically be much ___ than in a mean projection.
  3. If you have an unsigned 8-bit image with dimensions x=10, y=10, z=5; the highest value that you can possibly get in a maximum projection along the z axis is ___?
  4. Same image as above, the highest value you could possibly get in a sum projection along the z axis is ___?
  5. Same image as above, the highest value you could possibly get in a sum projection along the x axis is ___?
  6. Same image as above, the highest value you could possibly get in a mean projection along the y axis is ___?

Solution

  1. decreases
  2. larger
  3. 255 (highest value in a unsigned 8-bit image)
  4. 5 * 255 = 1275
  5. 10 * 255 = 2550
  6. 10 * 255 / 10 = 255

True or False

  1. Image projections are always along the z-axis.
  2. The data type of the projected image must be the same as the data type of the original image.

Solution

  1. False, you can project along any axis.
  2. False, in sum projections the pixel values are larger than in the original data and a different data type might be needed to represent them. For maximum projections however the data type needs not be changed. For mean projections it depends on the accuracy your science requires (decimal places need a floating point data type, 32-bit in IJ).

Explanations




Follow-up material

Recommended follow-up modules:

  • Volume rendering (TODO)

Learn more: