ConvpaintModel Class

ConvpaintModel Class#

The ConvpaintModel class is the core of Convpaint and combines feature extraction with pixel classification. It is the main class used for both the Convpaint napari plugin and the Convpaint API. This setup allows for a similar workflow in both environments and lets the user transition between them seamlessly.

Each ConvpaintModel instance in essence consists of three components:

a feature extractor model, defined in a separate class
a classifier, which is responsible for the final pixel classification
a Param object, which defines the details of the model procedures, and is also defined in a separate class

`init(alias=None, model_path=None, param=None, fe_name=None, **kwargs)`

Initializes a Convpaint model. This can be done with an alias, a model path, a param object, or a feature extractor name. If initialized by FE name, also other parameters can be given to override the defaults of the FE model.

If neither an alias, a model path, a param object, nor a feature extractor name is given, a default Convpaint model is created (defined in the get_default_params() method).

The four ways for initialization in detail:

By providing an alias (e.g., "vgg-s", "dino", "gaussian", etc.), in which case a corresponding model configuration will be loaded.
By providing a saved model path (model_path) to load a model defined earlier. This can be a .pkl file (holding the FE model, classifier and Param object) or .yml file (only defining the model parameters).
By providing a Param object, which contains model parameters.
By providing the name of the feature extractor and other parameters such as GPU usage, and feature extraction layers, in which case these kwargs will override the defaults of the feature extractor model.

Parameters:

Name	Type	Description	Default
`alias`	`str`	Alias of a predefined model configuration	`None`
`model_path`	`str`	Path to a saved model configuration	`None`
`param`	`Param`	Param object with the model parameters (see Param class for details)	`None`
`fe_name`	`str`	Name of the feature extractor model	`None`
`**kwargs`	`additional parameters`	Additional parameters to override defaults for the model or feature extractor (see Param class for details)	`{}`

Raises:

Type	Description
`ValueError`	If more than one of alias, model_path, param, or fe_name is provided.

`get_fe_models_types()` `staticmethod`

Returns a dictionary of all available feature extractors; names as keys and types as values.

Returns:

Name	Type	Description
`models_dict`	`dict`	Dictionary of all available feature extractors; names as keys and types as values.

`get_default_params()` `staticmethod`

Returns a param object, which defines the default Convpaint model.

Returns:

Name	Type	Description
`def_param`	`Param`	Param object with the default Convpaint model parameters.

`get_param(key)`

Returns the value of the given parameter key.

Parameters:

Name	Type	Description	Default
`key`	`str`	Key of the parameter to get	required

Returns:

Name	Type	Description
`param_value`	`any`	Value of the parameter with the given key

`get_params()`

Returns all parameters of the model in form of a copy of the Param object.

Returns:

Name	Type	Description
`param`	`Param`	Copy of the Param object with all parameters of the model

`set_param(key, val, ignore_warnings=False)`

Sets the value of the given parameter key.

Raises a warning if key FE parameters are set (intended only for initiation), unless ignore_warnings is set to True.

Parameters:

Name	Type	Description	Default
`key`	`str`	Key of the parameter to set	required
`val`	`any`	Value to set the parameter to	required
`ignore_warnings`	`bool`	Whether to suppress the warning for setting FE parameters	`False`

`set_params(param=None, ignore_warnings=False, **kwargs)`

Sets the parameters, given either as a Param object or as keyword arguments.

Note that the model is not reset and no new FE model is created. If fe_name, fe_use_gpu, and fe_layers change, you should create a new ConvpaintModel.

Parameters:

Name	Type	Description	Default
`param`	`Param`	Param object with the parameters to set	`None`
`ignore_warnings`	`bool`	Whether to suppress the warning for setting FE parameters	`False`
`**kwargs`	`parameters as keyword arguments`	Parameters to set as keyword arguments (instead of a Param object)	`{}`

Raises:

Type	Description
`ValueError`	If both a Param object and keyword arguments are provided.

`save(model_path, create_pkl=True, create_yml=True)`

Saves the model to a pkl and/or yml file.

The pkl file includes the classifier and the param object, as well as the features and targets (when using memory_mode). The yml file includes only the parameters of the model.

Note: Loading a saved model is only intended at model initiation.

Parameters:

Name	Type	Description	Default
`model_path`	`str`	Path to save the model to.	required
`create_pkl`	`bool`	Whether to create a pkl file	`True`
`create_yml`	`bool`	Whether to create a yml file	`True`

`create_fe(name, use_gpu=None, layers=None)` `staticmethod`

Creates a feature extractor model based on the given parameters.

Importantly, the feature extractor model is returned, but not saved in the model. If you want to initialize a ConvpaintModel with this feature extractor, create a new instance of ConvpaintModel and specify the feature extractor parameters.

Distinguishes between different types of feature extractors such as Hookmodels and initializes them accordingly.

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the feature extractor model	required
`use_gpu`	`bool`	Whether to use GPU for the feature extractor	`None`
`layers`	`list[str]`	List of layer names to extract features from	`None`

Returns:

Name	Type	Description
`fe_model`	`FeatureExtractor`	The created feature extractor model

`get_fe_defaults()`

Returns the default parameters of the feature extractor.

Where they are not sepecified by the feature extractor model, the default Convpaint params are used.

Returns:

Name	Type	Description
`new_param`	`Param`	Conpaint model defaults adjusted to the feature extractor defaults

`get_fe_layer_keys()`

Returns the keys of the feature extractor layers (None if the model uses no layers).

Returns:

Name	Type	Description
`keys`	`list[str] or None`	List of keys of the feature extractor layers, or None if the model uses no layers

`train(image, annotations, memory_mode=False, img_ids=None, use_rf=False, allow_writing_files=False, in_channels=None, skip_norm=False)`

Trains the Convpaint model's classifier given images and annotations.

Uses the FeatureExtractor model to extract features according to the criteria specified in the Param object. Then uses the features of annotated pixels to train the classifier.

Trains the internal classifier, which is also returned.

Parameters:

Name	Type	Description	Default
`image`	`ndarray or list[ndarray]`	Image to train the classifier on or list of images	required
`annotations`	`ndarray or list[ndarray]`	Annotations to train the classifier on or list of annotations Image and annotation lists must correspond to each other	required
`memory_mode`	`bool`	Whether to use memory mode If True, the annotations are registered and updated, and only features for new pixels are extracted	`False`
`img_ids`	`str or list[str]`	Image IDs to register the annotations with (when using memory_mode)	`None`
`use_rf`	`bool`	Whether to use a RandomForestClassifier instead of a CatBoostClassifier	`False`
`allow_writing_files`	`bool`	Whether to allow writing files for the CatBoostClassifier	`False`
`in_channels`	`list[int]`	List of channels to use for training	`None`
`skip_norm`	`bool`	Whether to skip normalization of the images before training If True, the images are not normalized according to the parameter `normalize` in the model parameters	`False`

Returns:

Type	Description
`clf : CatBoostClassifier or RandomForestClassifier`	Trained classifier (also saved inside the model instance)

`segment(image, in_channels=None, skip_norm=False, use_dask=False)`

Segments images by predicting the most probable class of each pixel using the trained classifier.

Uses the feature extractor model to extract features from the image. Predicts the classes of the pixels in the image using the trained classifier.

Parameters:

Name	Type	Description	Default
`image`	`ndarray or list[ndarray]`	Image to segment or list of images	required
`in_channels`	`list[int]`	List of channels to use for segmentation	`None`
`skip_norm`	`bool`	Whether to skip normalization of the images before segmentation If True, the images are not normalized according to the parameter `normalize` in the model parameters	`False`
`use_dask`	`bool`	Whether to use dask for parallel processing	`False`

Returns:

Name	Type	Description
`seg`	`ndarray or list[ndarray]`	Segmented image or list of segmented images (according to the input) Dimensions are equal to the input image(s) without the channel dimension

`predict_probas(image, in_channels=None, skip_norm=False, use_dask=False)`

Predicts the probabilities of the classes of the pixels in an image using the trained classifier.

Uses the feature extractor model to extract features from the image. Estimates the probability of each class based on the features and the trained classifier.

Parameters:

Name	Type	Description	Default
`image`	`ndarray or list[ndarray]`	Image to predict probabilities for or list of images	required
`in_channels`	`list[int]`	List of channels to use for prediction	`None`
`skip_norm`	`bool`	Whether to skip normalization of the images before prediction If True, the images are not normalized according to the parameter `normalize` in the model parameters	`False`
`use_dask`	`bool`	Whether to use dask for parallel processing	`False`

Returns:

Name	Type	Description
`probas`	`ndarray or list[ndarray]`	Predicted probabilities of the classes of the pixels in the image or list of images Dimensions are equal to the input image(s) without the channel dimension, with the class dimension added first

`get_feature_image(data, annotations=None, restore_input_form=True, memory_mode=False, img_ids=None, in_channels=None, skip_norm=False)`

Returns the features of images extracted by the feature extractor model.

Extract (and optionally reshape) feature images.

Processing pipeline:

Prepare dims -> list of [C,Z,H,W]
Optional channel subset
Optional normalization
Down/upsample (param.image_downsample)
Register/update annotations (memory_mode)
Padding (if needed)
(If annotations) extract planes / optionally tile
FE extraction
Optional reshaping: If restore_input_form & not annotations -> reshape + remove base padding + remove patch-multiple pad (via _restore_shape)
Restore original dimension semantics (drop singleton Z when appropriate)

Returns either a single feature array or lists (and optionally annotations / coords when restore_input_form=False).

Parameters:

Name	Type	Description	Default
`data`	`ndarray or list[ndarray]`	Image(s) to extract features from	required
`annotations`	`ndarray or list[ndarray]`	Annotations to extract features from. Image and annotation lists must correspond to each other.	`None`
`restore_input_form`	`bool`	Whether to restore the input form of the features. If True, no annotations are returned. If False, features are returned in their original form if no annotations are given, or paired with the processed annotations if they are given.	`True`
`memory_mode`	`bool`	Whether to use memory mode. If True, the annotations are registered and updated, and only features for new pixels are extracted.	`False`
`img_ids`	`str or list[str]`	Image IDs to register the annotations with (when using memory_mode)	`None`
`in_channels`	`list[int]`	List of channels to use for feature extraction	`None`
`skip_norm`	`bool`	Whether to skip normalization of the images before feature extraction. If True, the images are not normalized according to the parameter `normalize` in the model parameters.	`False`

Returns:

Name	Type	Description
`features`	`ndarray or list[ndarray]`	Extracted features of the image(s) or list of features for each image if input is a list. Features dimension is added first.
`annotations`	`(ndarray or list[ndarray], optional)`	Processed annotations of the image(s) or list of images, if annotations are given.

`reset_classifier()`

Resets the classifier of the model.

Resets the classifier object to None
Resets possible path to the classifier saved in the Param object
Resets the training features used with memory_mode

`reset_training()`

Resets the features and targets of the model which are used to iteratively train the model with the option memory_mode.

ConvpaintModel Class

ConvpaintModel Class#

__init__(alias=None, model_path=None, param=None, fe_name=None, **kwargs)

get_fe_models_types() staticmethod

get_default_params() staticmethod

get_param(key)

get_params()

set_param(key, val, ignore_warnings=False)

set_params(param=None, ignore_warnings=False, **kwargs)

save(model_path, create_pkl=True, create_yml=True)

create_fe(name, use_gpu=None, layers=None) staticmethod

get_fe_defaults()

get_fe_layer_keys()

train(image, annotations, memory_mode=False, img_ids=None, use_rf=False, allow_writing_files=False, in_channels=None, skip_norm=False)

segment(image, in_channels=None, skip_norm=False, use_dask=False)

predict_probas(image, in_channels=None, skip_norm=False, use_dask=False)

get_feature_image(data, annotations=None, restore_input_form=True, memory_mode=False, img_ids=None, in_channels=None, skip_norm=False)

reset_classifier()

reset_training()

`init(alias=None, model_path=None, param=None, fe_name=None, **kwargs)`

`get_fe_models_types()` `staticmethod`

`get_default_params()` `staticmethod`

`get_param(key)`

`get_params()`

`set_param(key, val, ignore_warnings=False)`

`set_params(param=None, ignore_warnings=False, **kwargs)`

`save(model_path, create_pkl=True, create_yml=True)`

`create_fe(name, use_gpu=None, layers=None)` `staticmethod`

`get_fe_defaults()`

`get_fe_layer_keys()`

`train(image, annotations, memory_mode=False, img_ids=None, use_rf=False, allow_writing_files=False, in_channels=None, skip_norm=False)`

`segment(image, in_channels=None, skip_norm=False, use_dask=False)`

`predict_probas(image, in_channels=None, skip_norm=False, use_dask=False)`

`get_feature_image(data, annotations=None, restore_input_form=True, memory_mode=False, img_ids=None, in_channels=None, skip_norm=False)`

`reset_classifier()`

`reset_training()`