Working with RGB images

The plugin handles directly RGB images as multi-channel images. RGB images are opened in a single layer withouth a slider to browse in napari. The three channels are used like those of actual stacks in the plugin, i.e. features are extracted from each channel separately and then used all together for training.

Here for example we segmented the pattern of the floor in the Main Building of University of Bern, where the plugin has been developed.

The model is not limited to background and foreground: multiple classes can be annotated and predicted.

import napari
from napari_convpaint.conv_paint_widget import ConvPaintWidget
from napari.utils import nbscreenshot
import matplotlib.pyplot as plt
import skimage

im = skimage.io.imread('../images/HB_floor_crop.tiff')

viewer = napari.Viewer()
viewer.add_image(im)
self = ConvPaintWidget(viewer)
viewer.window.add_dock_widget(self);

More classes

In this other example we segment 5 different classes of a more complex floor pattern. The training and prediction for more classes might take a little bit longer, but the prediction is excellent considering the very small number of annotated pixels.

To speed up the process there are different approaches, as for example downsampling the image, which results in a loss of prediction accuracy, or using GPU processing. For more details, please refer to the “In-depth explanations”