torchimage.cfa.utils package¶
Submodules¶
torchimage.cfa.utils.bayer_1c_4c module¶
Conversion between 2 alternative representations of a Bayer array.
(..., 1, 2h, 2w)
(..., 4, h, w)
where the 4 color channels are separatedNotice that the depth dimension (number of color channels) is always on axis -3
. Negative indexing is more
compatible with different dataset shapes.
A single bayer array under 1-channel representation is a (1, 2h, 2w)
tessellation like the one below, where
ABCD
can be any of the RGGB
, BGGR
, GRBG
, GBRG
sensor alignments:
ABABAB
CDCDCD
ABABAB
CDCDCD
When converted to 4-channel representation, the same bayer array will be as follows (stack of 4 2D arrays):
AAA BBB CCC DDD
AAA BBB CCC DDD
The process is deterministic and fully invertible (doesn’t involve any loss of information).
See also
Demosaic
- torchimage.cfa.utils.bayer_1c_4c.bayer_1c_to_4c(x: torch.Tensor)¶
Convert 1-channel bayer array to 4-channel bayer array.
Input shape is automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input 1-channel Bayer array of shape
(..., 1, h, w)
- Returns
y – Output 4-channel Bayer array of shape
(..., 4, h//2, w//2)
Has the same dtype and device as the input tensor.
- Return type
torch.Tensor
- torchimage.cfa.utils.bayer_1c_4c.bayer_4c_to_1c(x: torch.Tensor)¶
Convert 4-channel bayer array to 1-channel bayer array.
Input shape is automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input 4-channel Bayer array of shape
(..., 4, h, w)
- Returns
y – Output 1-channel Bayer array of shape
(..., 1, 2*h, 2*w)
Has the same dtype and device as the input tensor.
- Return type
torch.Tensor
torchimage.cfa.utils.bayer_to_rgb module¶
Lightweight, deterministic methods to convert bayer images to RGB images.
Currently, only downsampling is implemented. A bayer image of shape (..., 1, 2h, 2w)
or (..., 4, h, w)
will be
converted to an RGB image of shape (..., 3, h, w)
. We condense every 2-by-2 block of bayer detector results into
a single pixel, taking R pixel’s value as the new R, B pixel’s value as the new B, and the average of 2 G pixel’s values
as the new G.
For more advanced demosaicing algorithms (such as nearest neighbor, gradient-corrected bilinear interpolation, or
neural networks), please refer to other modules in demosaic
.
- torchimage.cfa.utils.bayer_to_rgb.downsample_1c_bayer_to_rgb(x: torch.Tensor, sensor_alignment: str)¶
Downsample a 1-channel bayer image to an RGB image.
Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input 1-channel bayer array of shape
(..., 1, 2h, 2w)
sensor_alignment (str) – Sensor alignment / bayer pattern
- Returns
Output 3-channel RGB array of shape
(..., 3, h, w)
.Note that the output height and width are halved.
- Return type
torch.Tensor
- torchimage.cfa.utils.bayer_to_rgb.downsample_4c_bayer_to_rgb(x: torch.Tensor, sensor_alignment: str)¶
Downsample a 4-channel bayer image to an RGB image.
Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input 4-channel bayer array of shape
(..., 4, h, w)
sensor_alignment (str) – Sensor alignment / bayer pattern
- Returns
Output 3-channel RGB array of shape
(..., 3, h, w)
.- Return type
torch.Tensor
torchimage.cfa.utils.rgb_to_bayer module¶
Convert RGB images to bayer images by adding mosaic.
For every pixel, 2 out of the 3 colors will be discarded while only 1 remains. Which color remains depends on the sensor alignment specified by the parameters. Therefore, this process is:
1. Irreversible: Since converting RGB to bayer involves a loss of information, its inverse (demosaicing) is a much harder problem. 2. Deterministic
This module works with generalized nchw format.
We separate converting to 1-channel bayer and 4-channel bayer into 2 functions. This is because conversion from RGB to bayer, as well as conversion between different bayer formats, all requires creating new tensors in memory. 2-step conversion (e.g. RGB to 1c bayer, then to 4c bayer) will bear the overhead from assigning and releasing memory for intermediate tensors.
See also
MatLab
- torchimage.cfa.utils.rgb_to_bayer.rgb_to_1c_bayer(x: torch.Tensor, sensor_alignment: str)¶
Convert RGB image to 1-channel bayer image.
Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input RGB image of shape
(..., 3, h, w)
, whereh
andw
must be evensensor_alignment (str) – Sensor alignment / bayer pattern
- Returns
y – Output 1-channel bayer image of shape
(..., 1, h, w)
.Has the same dtype and device as input.
- Return type
torch.Tensor
- torchimage.cfa.utils.rgb_to_bayer.rgb_to_4c_bayer(x: torch.Tensor, sensor_alignment: str)¶
Convert RGB image to 4-channel bayer image.
Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input RGB image of shape
(..., 3, h, w)
, whereh
andw
must be evensensor_alignment (str) – Sensor alignment / bayer pattern
- Returns
y – Output 4-channel bayer image of shape
(..., 4, h//2 , w//2)
.Has the same dtype and device as input.
- Return type
torch.Tensor
torchimage.cfa.utils.validation module¶
This module contains various validation tools for bayer and rgb images.
The validation tools deal with generalized nchw format: (number of samples, color channels, height, width). The number of samples can span multiple dimensions, such as (number of batches, number of samples in each batch, …).
- torchimage.cfa.utils.validation.check_1c_bayer(x: torch.Tensor)¶
Check if the input consists of 1-channel bayer arrays
- Parameters
x (torch.Tensor) – Should be 1-channel Bayer array of shape
(..., 1, h, w)
- Raises
AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.
- torchimage.cfa.utils.validation.check_4c_bayer(x: torch.Tensor)¶
Check if the input consists of 4-channel bayer arrays
- Parameters
x (torch.Tensor) – Should be 4-channel Bayer array of shape
(..., 4, h, w)
- Raises
AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.
- torchimage.cfa.utils.validation.check_rgb(x: torch.Tensor)¶
Check if the input is a valid RGB image (or image batch) in generalized nchw format.
- Parameters
x (torch.Tensor) –
Should be a 3-channel array of shape
(..., 3, h, w)
.The color channels should follow the exact order of Red, Green, and Blue, but this order can’t and won’t be verified.
- Raises
AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.
- torchimage.cfa.utils.validation.check_sensor_alignment(sensor_alignment)¶
Check if the input is a valid bayer sensor alignment pattern.
There are only 4 possible bayer sensor alignment patterns: GBRG, GRBG, BGGR, RGGB.
- Parameters
sensor_alignment (str) – Should be one of “GBRG”, “GRBG”, “BGGR”, and “RGGB”. Lower case letters are allowed but discouraged.
- Returns
sensor_alignment – The sensor alignment itself if the validation is successful. Lower case letters will be automatically converted to upper case.
- Return type
str
- Raises
AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.
Module contents¶
This submodule contains tools for converting between 1-channel and 4-channel bayer images, converting RGB images to bayer images, and validating input tensors or keywords for such functionalities.
- torchimage.cfa.utils.bayer_1c_to_4c(x: torch.Tensor)¶
Convert 1-channel bayer array to 4-channel bayer array.
Input shape is automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input 1-channel Bayer array of shape
(..., 1, h, w)
- Returns
y – Output 4-channel Bayer array of shape
(..., 4, h//2, w//2)
Has the same dtype and device as the input tensor.
- Return type
torch.Tensor
- torchimage.cfa.utils.bayer_4c_to_1c(x: torch.Tensor)¶
Convert 4-channel bayer array to 1-channel bayer array.
Input shape is automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input 4-channel Bayer array of shape
(..., 4, h, w)
- Returns
y – Output 1-channel Bayer array of shape
(..., 1, 2*h, 2*w)
Has the same dtype and device as the input tensor.
- Return type
torch.Tensor
- torchimage.cfa.utils.check_1c_bayer(x: torch.Tensor)¶
Check if the input consists of 1-channel bayer arrays
- Parameters
x (torch.Tensor) – Should be 1-channel Bayer array of shape
(..., 1, h, w)
- Raises
AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.
- torchimage.cfa.utils.check_4c_bayer(x: torch.Tensor)¶
Check if the input consists of 4-channel bayer arrays
- Parameters
x (torch.Tensor) – Should be 4-channel Bayer array of shape
(..., 4, h, w)
- Raises
AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.
- torchimage.cfa.utils.check_rgb(x: torch.Tensor)¶
Check if the input is a valid RGB image (or image batch) in generalized nchw format.
- Parameters
x (torch.Tensor) –
Should be a 3-channel array of shape
(..., 3, h, w)
.The color channels should follow the exact order of Red, Green, and Blue, but this order can’t and won’t be verified.
- Raises
AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.
- torchimage.cfa.utils.check_sensor_alignment(sensor_alignment)¶
Check if the input is a valid bayer sensor alignment pattern.
There are only 4 possible bayer sensor alignment patterns: GBRG, GRBG, BGGR, RGGB.
- Parameters
sensor_alignment (str) – Should be one of “GBRG”, “GRBG”, “BGGR”, and “RGGB”. Lower case letters are allowed but discouraged.
- Returns
sensor_alignment – The sensor alignment itself if the validation is successful. Lower case letters will be automatically converted to upper case.
- Return type
str
- Raises
AssertionError – Whenever the input fails a validation criterion. The assertion line itself should be helpful enough.
- torchimage.cfa.utils.downsample_1c_bayer_to_rgb(x: torch.Tensor, sensor_alignment: str)¶
Downsample a 1-channel bayer image to an RGB image.
Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input 1-channel bayer array of shape
(..., 1, 2h, 2w)
sensor_alignment (str) – Sensor alignment / bayer pattern
- Returns
Output 3-channel RGB array of shape
(..., 3, h, w)
.Note that the output height and width are halved.
- Return type
torch.Tensor
- torchimage.cfa.utils.downsample_4c_bayer_to_rgb(x: torch.Tensor, sensor_alignment: str)¶
Downsample a 4-channel bayer image to an RGB image.
Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input 4-channel bayer array of shape
(..., 4, h, w)
sensor_alignment (str) – Sensor alignment / bayer pattern
- Returns
Output 3-channel RGB array of shape
(..., 3, h, w)
.- Return type
torch.Tensor
- torchimage.cfa.utils.rgb_to_1c_bayer(x: torch.Tensor, sensor_alignment: str)¶
Convert RGB image to 1-channel bayer image.
Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input RGB image of shape
(..., 3, h, w)
, whereh
andw
must be evensensor_alignment (str) – Sensor alignment / bayer pattern
- Returns
y – Output 1-channel bayer image of shape
(..., 1, h, w)
.Has the same dtype and device as input.
- Return type
torch.Tensor
- torchimage.cfa.utils.rgb_to_4c_bayer(x: torch.Tensor, sensor_alignment: str)¶
Convert RGB image to 4-channel bayer image.
Input shape and sensor alignment are automatically validated, so it’s not necessary to check again outside of the scope.
- Parameters
x (torch.Tensor) – Input RGB image of shape
(..., 3, h, w)
, whereh
andw
must be evensensor_alignment (str) – Sensor alignment / bayer pattern
- Returns
y – Output 4-channel bayer image of shape
(..., 4, h//2 , w//2)
.Has the same dtype and device as input.
- Return type
torch.Tensor