Volume rendering

Prerequisites

Before starting this lesson, you should be familiar with:

• Volume projections

• Volume slicing

Learning Objectives

After completing this lesson, learners should be able to:

• Understand the concepts and some methodes of 3-D rendering.

• Appreciate that 3-D rendering can be challenging for some data.

• Perform basic volume rendering using a software tool.

Motivation

Intuitively grasping 3-D shapes requires visualisation of the whole object. This is not possible when just looking at one or several slices of a 3-D data set. Thus is it important about different volume rendering techniques that can create a 3-D appearance of the whole image. This is especially useful for sparse data, where individual 2-D slices only contain a small subset of the relevant information.

Concept map

Figure

Activities

Volume rendering

• Open a volume viewer with one of the below example images.
• Explore different volume rendering modes, such as
  • Maximum projection
    • Explore different camera positions
  • Volume rendering
    • Explore various transparency (alpha) mappings.
      • Interesting for 3-D FIB-SEM
  • Iso-surface rendering
    • Explore various surface thresholds
    • Explore changing the light position
• Save a snapshot.
• Create and save a custom animation, e.g. rotating the image.

Example data

• 3-D+t Chromosome congression
  • Useful to see in 3-D rendering (as it is very hard to see what is going on in 2-D slices)
• 3-D EM and segmentation
  • EM data is difficult to render in 3-D but for the segmentation channel it is very useful
  • Note that the segmentation channel does not look good in a max-projection; actual volume rendering is much better.
• 3-D MRI head
  • Good to compare various rendering modes
• 3-D Organoid nuclei
  • Challenge: Outer nuclei occulde inner nuclei
• 3-D FIB-SEM
  • Challenge: Dense signal, background is bright
• 3-D Tissue segmentation label mask
  • Challenge: Epithelial tissue occludes inside

ImageJ 3D Viewer

• Open Fiji
• Open a 3D image of choice (see above for a list of example images)
• Plugins > 3D Viewer
• Explore rendering modes Edit > Display as
  • Volume: Volume rendering
    • Edit > Transfer function
      • Transparency: Channel: Alpha
  • Iso-Surface: Surface
    • Edit > Adjust threshold
    • Edit > Change color

skimage napari

###
# To create an animation of the volume the napari-animation plugin is needed.
# pip install napari-animation
###

import numpy as np
from skimage.io import imread
import napari

# Read the image
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyzt_8bit__starfish_chromosomes.tif')
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyzc_8bit__em_synapses_and_labels.tif')
image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_8bit_calibrated__mri_full_head.tif')
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_8bit_calibrated__organoid_nuclei.tif')
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_8bit_calibrated__fib_sem_crop.tif')
# image = imread('https://github.com/NEUBIAS/training-resources/raw/master/image_data/xyz_8bit_calibrated_labels__platy_tissues.tif')

# Check image type and values
print(image.dtype)
print(np.min(image), np.max(image))
print(image.shape)

# Instantiate the napari viewer
viewer = napari.Viewer()

# View the intensity image as grayscale
viewer.add_image(image, name='image', colormap='gray')
# Napari GUI: choose a colormap according to the data type

# Napari GUI: change viewer from 2D to 3D, zoom in and out and rotate the volume
# Note: these values are optimized for xyz_8bit_calibrated__mri_full_head.tif
viewer.dims.ndisplay = 3
viewer.camera.zoom = 2
viewer.camera.angles = (0, -60, 90)

# Napari GUI: use rendering (and attenuation) modes
# Parameters can be changed for reproducibility
viewer.layers['image'].rendering = 'attenuated_mip'
viewer.layers['image'].attenuation = 1.

# Take a screenshot of the scene created
from napari.utils import nbscreenshot
nbscreenshot(viewer)

# Acquire the frame as numpy array and add it to the napari GUI
screenshot = viewer.screenshot()
viewer.add_image(screenshot, name='screenshot')
viewer.dims.ndisplay = 2

# Napari GUI: realize this is a 2D RGBA image and can be saved as a PNG for presentations
print(screenshot.dtype)
print(np.min(screenshot), np.max(screenshot))
print(screenshot.shape)

# Napari GUI: use napari-animation (https://github.com/napari/napari-animation) to create an animation of the volume

Assessment

True or False

• Surface rendering and volume rendering are identical words for the same 3-D visualisation method.
• Volume rendering is particularly useful for data containing very dense 3-D information such as very many cells or nuclei in an organ of a biological specimen.
• Volume rendering is a simple algorithm that can easily be used without expert knowledge.
• Volume rendering is very useful to get an impression of the morphology and spatial distribution of objects.

Solution

• False. Although both methods are used for 3-D rendering they are different. In surface rendering one needs to define “the shell” of an object and only this will be visible. In volume rendering the intensity of all voxels can be represented such as in a maximum intensity projection based volume rendering.
• False. If the data is very dense, there is a high probabilty that no matter from which angle you look there will be objects hidden behind other objects. Thus, sparse data can be more suited to 3-D rendering than very dense data.
• False. In fact, volume rendering is very complex and there are many things to learn to master it (see for example this website.
• True. If the sample is not too dense, volume rendering allows one to get a quick overview of the whole 3-D specimen and its morphology.

Explanations

Follow-up material

Recommended follow-up modules:

Learn more:

• Blender documentation

• Wikipedia: Volume rendering