Prerequisites

Before starting this lesson, you should be familiar with:

Learning Objectives

After completing this lesson, learners should be able to:
• Understand that images have a data type which limits the values that the pixels in the image can have.

• Understand common data types such as 8-bit, 12-bit and 16-bit unsigned integer.

Motivation

Images contain numerical values that must be somehow stored on the hard disc or within the computer memory. To do so, for each pixel a certain amount of space (memory) must be allocated (usually measure in bits). Generally, the more bits you allocate, the bigger are the numbers that you can store, however, you also need more space. Thus choosing the right data type usually is a balance between what you can represent and how much space you want to afford for this. Especially, for large image data such as volume EM and light-sheet data, the choice of the data type can have quite some impact on your purse. In addition, certain operations on images can yield results with values outside of the original data type; this is a serious and frequently occurring source of mistakes when handling image data and thus must be well understood!

Concept map

graph TD I("Image") -->|has|DT("Data type") DT -->|limits|PV("Pixel values") DT -->|has|BD("Bit depth") DT -->|has|VR("Value range")

Activities

Find saturated pixels in an 8-bit image

Saturation, i.e. pixel value at the upper end of the datatype, is a typical problem in fluorescence microscopy images.

Show activity for:

skimage napari

Find saturated pixels in an 12-bit image

Saturation, i.e. pixel value at the upper end of the datatype, is a typical problem in fluorescence microscopy images.

Inspecting 12-bit images, as acquired by some camera based systems, is particularly tricky, because 12-bit images are typically represented as 16-bit images, both on disk and within analysis software.

• Open xy_12bit__saturated_plant.tif
• Observe that while this opens as 16-bit image there are many pixels at the highest value of 4095, which is the maximum of a 12-bit image.

Show activity for:

skimage napari

Inspect a binary image

This activity shows that correctly handling binary images can be tricky because there typically is no dedicated binary datatype for storing images on disk.

• Open xy_8bit_binary__h2b.tif
• Observe that this an unsigend 8-bit integer image, where only two of the possible 256 gray values used. Thus, in practice this probably represents a binary image.

Show activity for:

skimage napari

Inspect a ScanR microscope 12-bit image

This activity shows that correctly handling 12-bit data can be tricky, becuause typically neither on disk nor in memory there is a 12-bit datatype.

• Open xy_16bit__scanR_datatype_issue.tif
• For all software that we tested this image is opened as an unsigned integer 16-bit image with a value range from 32963 to 36863
• In fact, this image is a 12-bit image that has an offset of 2^15, probably due to a misinterpretation of the first bit
• 2^15 = 32768 (minimal possible value, which does not occur in this particular image because there is some background)
• 2^15 + 2^12 - 1 = 36863 (maximal value possible in this image/datatype)
• Check how many saturated pixels there are

Show activity for:

skimage napari

Explore various image data types

Open the following images and discuss their data type and whether there are any signs of intensity clipping.

• xy_8bit__nuclei_noisy_different_intensity.tif
• Appreciate that this image has no major issues.
• xy_8bit__nuclei_intensity_clipping_issue_a.tif
• This image has saturation issues (many pixels of value 255).
• This is problematic as one cannot compare the intensity of some of the nuclei.
• xy_8bit__nuclei_intensity_clipping_issue_b.tif
• This image clips intensities at low gray scale levels (many zeros in the image).
• This is problematic as the data could be seen as if there is no DNA within some places of the nuclei, which most likely would be the wrong biophysical interpretation.
• xy_8bit_binary__h2b.tif
• Even though this is an unsigend 8-bit integer image there are only two of the possible 256 gray values used. Thus, in practice this probably represents a binary image.
• xy_16bit__autophagosomes.tif
• This is an unsigned integer 16-bit image.
• Not all 65535 gray values are used.
• In fact, the highest value in the image is 1200 which is less than 4095, thus, maybe this image was actually acquired with a 12-bit camera or some other bit-depth less than 16.
• In general, working with bit-depths between 8 and 16 is a bit problematic because there is not much support in terms of file formats and analysis software. Thus in the process one may loose track of the original bit-depth of the image.
• xy_16bit__scanR_datatype_issue.tif
• This image is stored as an unsigned integer 16-bit image.
• The range of the data is from 32963 to 36863
• In fact, this image is a 12-bit image that has an offset of 2^15.
• 2^15 = 32768 (minimal possible value, which does not occur in this particular image because there is some background)
• 2^15 + 2^12 - 1 = 36863 (maximal value possible in this image)
• This example, which is produced by a real commercial microscope, again demonstrates that interpreting 16 bit images can be tricky, as they may actually be of a lower bit depth.

Show activity for:

ImageJ GUI

• For each image mentioned in the activity perform the below tasks.
• Open the image in Fiji.
• Use various ways to inspect the image and verify the comments that are given below the respective image in the activity.
• To this end, useful tools are:
• Image › Show Info...
• Inspect pixel values by hovering over the image with the mouse.
• Analyze › Histogram
• Analyze › Plot Profile

Explore more image data types and their metadata

Observe that for some software the datatype of the loaded image does not match the datatype given in the metadata.

The reason is that some software only support data types where the bit depth is a multiple of 8. For example, unsigned integer 12-bit data may not be supported.

This is very important as you may misinterpret whether your image contains saturated pixels or not.

Image data

Show activity for:

ImageJ GUI

• Open the image using Plugins > Bio-Formats > Bio-Formats Importer
• Select [X] Display OME-XML metadata
• Click [ OK ]
• Check whether the information in Image > Type is the same as the one mentioned in the displayed metadata (look for SignificantBits and Type)
• Also check the maximum value in the image, e.g. using Analyze > Histogram
• How does this maximum value compare to the image datatype?
• For example, you may find a value of 4095, which is the maximum of an unsigned integer 12-bit image, which may be the datatype mentioned in the image metadata, however ImageJ may represent this image as a 16-bit image. Appreciate that this can be confusing!
• If you find the maximum of the image to be identical to maximum that the datatype of the image can represent you may have an issue with saturation! Check this
• by hovering with the mouse over bright regions
• using the HiLo LUT with appropriate contrast settings, i.e. the maximum should be the maximum of your datatype!

Assessment

True or false? Discuss with your neighbor!

1. Changing pixel data type never changes pixel values.
2. Converting from 16-bit unsigned integer to 32-bit floating point never changes the pixel values.
3. Changing from 32-bit floating point to 16-bit unsigned integer never changes the pixel values.
4. There is only one correct way to convert from 16-bit to 8-bit.
5. If the highest value in an image is 255, one can conclude that it is an 8-bit unsigned integer image.
6. If the highest value in an image is 1034, one can conclude that it is not an 8-bit unsigned integer image.
7. If the bit-depth is 16 and there are a lot of neighboring pixels with the value 4095 and no pixels with a higher value, most likely this image was acquired with 12-bit camera.

1. False
2. True
3. False
4. False
5. False
6. True
7. True

Explanations

Image data types

The pixels in an image have a certain data type. The data type limits the values that pixels can take.

For example, unsigned N-bit integer images can represent values from 0 to 2^N -1, e.g.

• 8-bit unsigned integer: 0 - 255
• 12-bit unsigned integer: 0 - 4095
• 16-bit unsigned integer: 0 - 65535

Intensity clipping (saturation)

If the value of a pixel in an N-bit unsigned integer image is equal to either 0 or 2^N - 1, you cannot know for sure whether you lost information at some point during the image acquisition or image storage. For example, if there is a pixel with the value 255 in an unsigned integer 8-bit image, it may be that the actual intensity “was higher”, e.g. would have corresponded to a gray value of 302. One speaks of “saturation” or “intensity clipping” in such cases. It is important to realise that there can be also clipping at the lower end of the range (some microscopes have an unfortunate “offset” slider that can be set to negative values, which can cause this).

Follow-up material

Recommended follow-up modules: