torchimage.cfa.utils package

Submodules

torchimage.cfa.utils.bayer_1c_4c module

Conversion between 2 alternative representations of a Bayer array.

1-channel representation: (..., 1, 2h, 2w)
4-channel representation: (..., 4, h, w) where the 4 color channels are separated

Notice that the depth dimension (number of color channels) is always on axis -3. Negative indexing is more compatible with different dataset shapes.

A single bayer array under 1-channel representation is a (1, 2h, 2w) tessellation like the one below, where ABCD can be any of the RGGB, BGGR, GRBG, GBRG sensor alignments:

ABABAB
CDCDCD
ABABAB
CDCDCD

When converted to 4-channel representation, the same bayer array will be as follows (stack of 4 2D arrays):

AAA BBB CCC DDD
AAA BBB CCC DDD

The process is deterministic and fully invertible (doesn’t involve any loss of information).

See also

Demosaic

torchimage.cfa.utils.bayer_1c_4c.bayer_1c_to_4c(x: torch.Tensor)

Convert 1-channel bayer array to 4-channel bayer array.

Input shape is automatically validated, so it’s not necessary to check again outside of the scope.

Parameters

x (torch.Tensor) – Input 1-channel Bayer array of shape (..., 1, h, w)

Returns

y – Output 4-channel Bayer array of shape (..., 4, h//2, w//2)

Has the same dtype and device as the input tensor.

Return type

torch.Tensor

torchimage.cfa.utils.bayer_1c_4c.bayer_4c_to_1c(x: torch.Tensor)

Convert 4-channel bayer array to 1-channel bayer array.

Input shape is automatically validated, so it’s not necessary to check again outside of the scope.

Parameters

x (torch.Tensor) – Input 4-channel Bayer array of shape (..., 4, h, w)

Returns

y – Output 1-channel Bayer array of shape (..., 1, 2*h, 2*w)

Has the same dtype and device as the input tensor.

Return type

torch.Tensor

torchimage.cfa.utils.bayer_to_rgb module

Lightweight, deterministic methods to convert bayer images to RGB images.

Currently, only downsampling is implemented. A bayer image of shape (..., 1, 2h, 2w) or (..., 4, h, w) will be converted to an RGB image of shape (..., 3, h, w). We condense every 2-by-2 block of bayer detector results into a single pixel, taking R pixel’s value as the new R, B pixel’s value as the new B, and the average of 2 G pixel’s values as the new G.

For more advanced demosaicing algorithms (such as nearest neighbor, gradient-corrected bilinear interpolation, or neural networks), please refer to other modules in demosaic.

torchimage.cfa.utils.bayer_to_rgb.downsample_1c_bayer_to_rgb(x: torch.Tensor, sensor_alignment: str)

Downsample a 1-channel bayer image to an RGB image.

Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.

Parameters
  • x (torch.Tensor) – Input 1-channel bayer array of shape (..., 1, 2h, 2w)

  • sensor_alignment (str) – Sensor alignment / bayer pattern

Returns

Output 3-channel RGB array of shape (..., 3, h, w).

Note that the output height and width are halved.

Return type

torch.Tensor

torchimage.cfa.utils.bayer_to_rgb.downsample_4c_bayer_to_rgb(x: torch.Tensor, sensor_alignment: str)

Downsample a 4-channel bayer image to an RGB image.

Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.

Parameters
  • x (torch.Tensor) – Input 4-channel bayer array of shape (..., 4, h, w)

  • sensor_alignment (str) – Sensor alignment / bayer pattern

Returns

Output 3-channel RGB array of shape (..., 3, h, w).

Return type

torch.Tensor

torchimage.cfa.utils.rgb_to_bayer module

Convert RGB images to bayer images by adding mosaic.

For every pixel, 2 out of the 3 colors will be discarded while only 1 remains. Which color remains depends on the sensor alignment specified by the parameters. Therefore, this process is:

1. Irreversible: Since converting RGB to bayer involves a loss of information, its inverse (demosaicing) is a much harder problem. 2. Deterministic

This module works with generalized nchw format.

We separate converting to 1-channel bayer and 4-channel bayer into 2 functions. This is because conversion from RGB to bayer, as well as conversion between different bayer formats, all requires creating new tensors in memory. 2-step conversion (e.g. RGB to 1c bayer, then to 4c bayer) will bear the overhead from assigning and releasing memory for intermediate tensors.

See also

MatLab

torchimage.cfa.utils.rgb_to_bayer.rgb_to_1c_bayer(x: torch.Tensor, sensor_alignment: str)

Convert RGB image to 1-channel bayer image.

Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.

Parameters
  • x (torch.Tensor) – Input RGB image of shape (..., 3, h, w), where h and w must be even

  • sensor_alignment (str) – Sensor alignment / bayer pattern

Returns

y – Output 1-channel bayer image of shape (..., 1, h, w).

Has the same dtype and device as input.

Return type

torch.Tensor

torchimage.cfa.utils.rgb_to_bayer.rgb_to_4c_bayer(x: torch.Tensor, sensor_alignment: str)

Convert RGB image to 4-channel bayer image.

Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.

Parameters
  • x (torch.Tensor) – Input RGB image of shape (..., 3, h, w), where h and w must be even

  • sensor_alignment (str) – Sensor alignment / bayer pattern

Returns

y – Output 4-channel bayer image of shape (..., 4, h//2 , w//2).

Has the same dtype and device as input.

Return type

torch.Tensor

torchimage.cfa.utils.validation module

This module contains various validation tools for bayer and rgb images.

The validation tools deal with generalized nchw format: (number of samples, color channels, height, width). The number of samples can span multiple dimensions, such as (number of batches, number of samples in each batch, …).

torchimage.cfa.utils.validation.check_1c_bayer(x: torch.Tensor)

Check if the input consists of 1-channel bayer arrays

Parameters

x (torch.Tensor) – Should be 1-channel Bayer array of shape (..., 1, h, w)

Raises

AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.

torchimage.cfa.utils.validation.check_4c_bayer(x: torch.Tensor)

Check if the input consists of 4-channel bayer arrays

Parameters

x (torch.Tensor) – Should be 4-channel Bayer array of shape (..., 4, h, w)

Raises

AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.

torchimage.cfa.utils.validation.check_rgb(x: torch.Tensor)

Check if the input is a valid RGB image (or image batch) in generalized nchw format.

Parameters

x (torch.Tensor) –

Should be a 3-channel array of shape (..., 3, h, w).

The color channels should follow the exact order of Red, Green, and Blue, but this order can’t and won’t be verified.

Raises

AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.

torchimage.cfa.utils.validation.check_sensor_alignment(sensor_alignment)

Check if the input is a valid bayer sensor alignment pattern.

There are only 4 possible bayer sensor alignment patterns: GBRG, GRBG, BGGR, RGGB.

Parameters

sensor_alignment (str) – Should be one of “GBRG”, “GRBG”, “BGGR”, and “RGGB”. Lower case letters are allowed but discouraged.

Returns

sensor_alignment – The sensor alignment itself if the validation is successful. Lower case letters will be automatically converted to upper case.

Return type

str

Raises

AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.

Module contents

This submodule contains tools for converting between 1-channel and 4-channel bayer images, converting RGB images to bayer images, and validating input tensors or keywords for such functionalities.

torchimage.cfa.utils.bayer_1c_to_4c(x: torch.Tensor)

Convert 1-channel bayer array to 4-channel bayer array.

Input shape is automatically validated, so it’s not necessary to check again outside of the scope.

Parameters

x (torch.Tensor) – Input 1-channel Bayer array of shape (..., 1, h, w)

Returns

y – Output 4-channel Bayer array of shape (..., 4, h//2, w//2)

Has the same dtype and device as the input tensor.

Return type

torch.Tensor

torchimage.cfa.utils.bayer_4c_to_1c(x: torch.Tensor)

Convert 4-channel bayer array to 1-channel bayer array.

Input shape is automatically validated, so it’s not necessary to check again outside of the scope.

Parameters

x (torch.Tensor) – Input 4-channel Bayer array of shape (..., 4, h, w)

Returns

y – Output 1-channel Bayer array of shape (..., 1, 2*h, 2*w)

Has the same dtype and device as the input tensor.

Return type

torch.Tensor

torchimage.cfa.utils.check_1c_bayer(x: torch.Tensor)

Check if the input consists of 1-channel bayer arrays

Parameters

x (torch.Tensor) – Should be 1-channel Bayer array of shape (..., 1, h, w)

Raises

AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.

torchimage.cfa.utils.check_4c_bayer(x: torch.Tensor)

Check if the input consists of 4-channel bayer arrays

Parameters

x (torch.Tensor) – Should be 4-channel Bayer array of shape (..., 4, h, w)

Raises

AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.

torchimage.cfa.utils.check_rgb(x: torch.Tensor)

Check if the input is a valid RGB image (or image batch) in generalized nchw format.

Parameters

x (torch.Tensor) –

Should be a 3-channel array of shape (..., 3, h, w).

The color channels should follow the exact order of Red, Green, and Blue, but this order can’t and won’t be verified.

Raises

AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.

torchimage.cfa.utils.check_sensor_alignment(sensor_alignment)

Check if the input is a valid bayer sensor alignment pattern.

There are only 4 possible bayer sensor alignment patterns: GBRG, GRBG, BGGR, RGGB.

Parameters

sensor_alignment (str) – Should be one of “GBRG”, “GRBG”, “BGGR”, and “RGGB”. Lower case letters are allowed but discouraged.

Returns

sensor_alignment – The sensor alignment itself if the validation is successful. Lower case letters will be automatically converted to upper case.

Return type

str

Raises

AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.

torchimage.cfa.utils.downsample_1c_bayer_to_rgb(x: torch.Tensor, sensor_alignment: str)

Downsample a 1-channel bayer image to an RGB image.

Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.

Parameters
  • x (torch.Tensor) – Input 1-channel bayer array of shape (..., 1, 2h, 2w)

  • sensor_alignment (str) – Sensor alignment / bayer pattern

Returns

Output 3-channel RGB array of shape (..., 3, h, w).

Note that the output height and width are halved.

Return type

torch.Tensor

torchimage.cfa.utils.downsample_4c_bayer_to_rgb(x: torch.Tensor, sensor_alignment: str)

Downsample a 4-channel bayer image to an RGB image.

Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.

Parameters
  • x (torch.Tensor) – Input 4-channel bayer array of shape (..., 4, h, w)

  • sensor_alignment (str) – Sensor alignment / bayer pattern

Returns

Output 3-channel RGB array of shape (..., 3, h, w).

Return type

torch.Tensor

torchimage.cfa.utils.rgb_to_1c_bayer(x: torch.Tensor, sensor_alignment: str)

Convert RGB image to 1-channel bayer image.

Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.

Parameters
  • x (torch.Tensor) – Input RGB image of shape (..., 3, h, w), where h and w must be even

  • sensor_alignment (str) – Sensor alignment / bayer pattern

Returns

y – Output 1-channel bayer image of shape (..., 1, h, w).

Has the same dtype and device as input.

Return type

torch.Tensor

torchimage.cfa.utils.rgb_to_4c_bayer(x: torch.Tensor, sensor_alignment: str)

Convert RGB image to 4-channel bayer image.

Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.

Parameters
  • x (torch.Tensor) – Input RGB image of shape (..., 3, h, w), where h and w must be even

  • sensor_alignment (str) – Sensor alignment / bayer pattern

Returns

y – Output 4-channel bayer image of shape (..., 4, h//2 , w//2).

Has the same dtype and device as input.

Return type

torch.Tensor