Volume rendering

Prerequisites

Before starting this lesson, you should be familiar with:

• Volume projections

• Volume slicing

Learning Objectives

After completing this lesson, learners should be able to:

• Understand the concepts and some methodes of 3-D rendering.

• Appreciate that 3-D rendering can be challenging for some data.

• Perform basic volume rendering using a software tool.

Motivation

Intuitively grasping 3-D shapes requires visualisation of the whole object. This is not possible when just looking at one or several slices of a 3-D data set. Thus is it important about different volume rendering techniques that can create a 3-D appearance of the whole image. This is especially useful for sparse data, where individual 2-D slices only contain a small subset of the relevant information.

Concept map

Figure

Activities

• Example data:
  • 3-D+t Chromosome congression
    • Useful to see in 3-D rendering (as it is very hard to see what is going on in 2-D slices)
  • 3-D EM and segmentation
    • EM data is difficult to render in 3-D but for the segmentation channel it is very useful
    • Note that the segmentation channel does not look good in a max-projection; actual volume rendering is much better.
  • 3-D MRI head
    • Good to compare various rendering modes
  • 3-D Organoid nuclei
    • Challenge: Outer nuclei occulde inner nuclei
  • 3-D FIB-SEM
    • Challenge: Dense signal, background is bright
  • 3-D Tissue segmentation label mask
    • Challenge: Epithelial tissue occludes inside
• Open a volume viewer.
• Explore different volume rendering modes (as available in your softwares):
  • Maximum projection
    • Explore different camera positions
  • Volume rendering
    • Explore various transparency (alpha) mappings.
      • Interesting for 3-D FIB-SEM
  • Iso-surface rendering
    • Explore various surface thresholds
    • Explore changing the light position
• Save a snapshot.
• Create and save a custom animation, e.g. rotating the image.

skimage napari

###
# To create an animation of the volume the napari-animation plugin is needed.
# pip install napari-animation
###

import numpy as np
from skimage.io import imread
import napari

# Read the image
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyzt_8bit__starfish_chromosomes.tif')
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyzc_8bit__em_synapses_and_labels.tif')
image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_8bit_calibrated__mri_full_head.tif')
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_8bit_calibrated__organoid_nuclei.tif')
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_8bit_calibrated__fib_sem_crop.tif')
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_8bit_calibrated_labels__platy_tissues.tif')

# Check image type and values
print(image.dtype)
print(np.min(image), np.max(image))
print(image.shape)

# Instantiate the napari viewer
viewer = napari.Viewer()

# View the intensity image as grayscale
viewer.add_image(image, name='image', colormap='gray')
# Napari GUI: choose a colormap according to the data type

# Napari GUI: change viewer from 2D to 3D, zoom in and out and rotate the volume
# Note: these values are optimized for xyz_8bit_calibrated__mri_full_head.tif
viewer.dims.ndisplay = 3
viewer.camera.zoom = 2
viewer.camera.angles = (0, -60, 90)

# Napari GUI: use rendering (and attenuation) modes
# Parameters can be changed for reproducibility
viewer.layers['image'].rendering = 'attenuated_mip'
viewer.layers['image'].attenuation = 1.

# Take a screenshot of the scene created
from napari.utils import nbscreenshot
nbscreenshot(viewer)

# Acquire the frame as numpy array and add it to the napari GUI
screenshot = viewer.screenshot()
viewer.add_image(screenshot, name='screenshot')
viewer.dims.ndisplay = 2

# Napari GUI: realize this is a 2D RGBA image and can be saved as a PNG for presentations
print(screenshot.dtype)
print(np.min(screenshot), np.max(screenshot))
print(screenshot.shape)

# Napari GUI: use napari-animation (https://github.com/napari/napari-animation) to create an animation of the volume

Assessment

True or False

• TODO.

Solution

• TODO

Explanations

Follow-up material

Recommended follow-up modules:

Learn more:

• Blender documentation

• Wikipedia: Volume rendering