Segmentation using a Shape Model

Initial threshold-based segmentation

It is first necessary to segment the object as well as possible using a simple threshold. Use the standard thresholding options in the draw task page to do this. You will be left with a set of contours defining the surface, perhaps quite accurately in places, but less accurately elsewhere.

Loading a shape model

Now switch to the "Fit shape model" task. Load a shape model in the form of a .ply file. The header of the file must include at least three comments of the form
comment FIT_MODEL_LM "cochlear apex" 2436
comment FIT_MODEL_LM "centre of oval window" 5567
comment FIT_MODEL_LM "p-a canal bifurcation" 6735
describing anatomical landmarks and the indices of the mesh vertices where they are located. These landmarks are used for a rough, initial alignment of the model to the data. If Stradwin finds, in the same folder, a file with the same name but with suffix .ssm (statistical shape model), it will load that too. .ssm files may be generated using wxRegSurf: they describe the principal modes of variation across a cohort of surfaces. One of Stradwin's nonrigid model-fitting algorithms allows the surface to deform only along the shape modes contained in the .ssm file.

Model initialization

The next step is to fit the model very roughly using three or more landmarks. Select the "mark" tool and click in the review canvas on each landmark in turn. A prompt reminds you which landmark to define next. After all landmarks have been defined, click "Fit to landmarks". You will see the red surface moving to approximately the right position, though it may require an additional reflection. You can toggle between reflected and unreflected fits by pressing the "Fit to landmarks" button repeatedly.

Alternatively, you can manipulate the model into roughly the right position by holding down the ctrl key while clicking and dragging in the 3D window. The left and middle mouse buttons rotate the surface, while the right button translates the surface. Press r in the 3D window to toggle between reflected and unreflected versions of the surface.

The rest of the process involves refining the fit using the contours defined in the first step. Select the appropriate object in the "Points from" menu and then click "Map points". You will see in the review canvas a lot of red points derived from the contours. We are going to refine the position of the surface so that it aligns as well as possible with the closest red points, using an iterative registration algorithm. Points that are currently closest to the surface are shown in bright red, other points in dark red.

Points are not coloured bright red if the surface normals of the model and the thresholded object are pointing in opposite directions. However, this requires correct inside/outside segmentation at the thresholding stage. If there has been an error, and points are not coloured bright red when they should be, tick the "Flip inside/outside" checkbox and then "Map points" again.

Iterative registration

Scroll through the segmented frames and edit the red points manually. You will need to add some points (use the "mark" tool) where the thresholded contours did not capture the surface correctly, and perhaps also delete some points (use the "erase" tool) where there are confusing red points on the wrong surface. The "Display compounding" slider allows you to see red points on nearby frames too, and delete them all with one sweep of the "erase" tool. At any time, you can refit the red surface to the current set of red points by pressing the "Start registration" button. For now, leave the "Nonrigid flexibility/Number of SSM modes" control at zero, and run 10 registration iterations at a time, before reviewing the result. With no nonrigid flexibility, the fit is rigid plus isotropic scaling (7 degrees of freedom).

The "Search range" slider sets how far away from the surface Stradwin is prepared to look for a red point. Initially, when the fit is poor, you will need quite a large value so that the registration algorithm can discover the correct red points and converge on a good solution. Later, you can reduce the slider value so that only nearby red points affect the registration process: all the other (dark red) points are ignored.

When you have a decent fit, and a decent set of bright red points to fit to (dark red points are less problematic), you can refine the fit by introducing some nonrigid deformation. If you did not load a .ssm file, a locally affine deformation (LAD) is used, controlled by the "Nonrigid flexibility" slider. If you did load a .ssm file, you may choose between LAD and the statistical shape model (SSM), selecting the "Number of SSM modes" to use with the slider. Start with a low "Nonrigid flexibility" or "Number of SSM modes". You may then need to manually refine the red points before doing a final fit with a higher "Nonrigid flexibility" or a higher number of SSM modes. If the surface gets tangled as you edit the red points, you can reset the nonrigid deformation by selecting zero "Number of SSM modes" or "Nonrigid flexibility", and then refitting the model.

Mapping back to contours

When you are happy with the fit, you can either save the fitted surface as a .ply or .wrl file, or generate a set of segmentation contours from the fitted model. Use the "Contours to" menu to select the object which will receive the segmentation result (this could be the same one you selected in the "Points from" menu) and press "Map contours". This generates a new set of contours for this object. You can choose whether to insert contours onto every frame or every n frames, counting from the current frame. You can also select the accuracy with which the contours should follow the fitted surface. Finally, by selecting "Low", you can generate contours only on frames 0 to the current frame. Or, by selecting "High", you can generate contours only on frames from the current frame to the final frame. This may be useful when the model represents only part of the surface, and you want to merge the model segmentation in some frames with the thresholded segmentation in other frames.

If any further editing is required, this can be performed using the standard tools on the draw task page.