Changing feature extractor (DINOv2) to use Convpaint for animal tracking

Changing feature extractor (DINOv2) to use Convpaint for animal tracking#

With the pretrained vision transformer DINOv2 as feature extractor, Convpaint is remarkable at detecting animal body parts - or even actions such as closing/opening eyes.

Here is a sample frame from a movie, showing segmentation of mouse head, tail and body:

Tracking animal body parts in movies#

To achieve this, the first step is to change the feature extractor from default VGG16 to DINOv2:

  1. Switch to the Model options tab.

  2. Select DINOv2 (dinov2_vits14_reg1) as feature extractor from the list.

  3. Click on Set feature extractor and you are good to go!

Annotate and train your Convpaint model as usual.

Tip: Select `Use GPU` to benefit from GPU acceleration! (Cuda on windows, or MPS on Mac devices.)

Example: Tracking shark body parts in a movie#

These are the scribble annotations provided for training:

And this is the resulting segmentation video:

Contents