lacss.tracking

KalmanFilter

Bases: Protocol

KalmanFilter Protocol definition

initialize(measurement)

Create track from unassociated measurement.

Parameters:

Name	Type	Description	Default
measurement		data in measurement space	required

Returns:

Name	Type	Description
mean	ndarray	Unobserved velocities are initialized to 0 mean.
cov	ndarray	covariance matrix (8x8 dimensional) of the new track.

predict(mean, covariance)

Run Kalman filter prediction step.

Parameters:

Name	Type	Description	Default
mean		mean vector of the object state at the previous time step.	required
covariance		The covariance matrix of the object state at the previous time step.	required

Returns:

Name	Type	Description
mean	ndarray	the mean vector of the predicted state
cov	ndarray	the covariance matrix of the predicted

update(mean, covariance, measurement)

Run Kalman filter correction step.

Parameters:

Name	Type	Description	Default
mean		The predicted state's mean vector.	required
covariance		The state's covariance matrix.	required
measurement		The measurement vector	required

Returns:

Name	Type	Description
mean	ndarray	mean vector of the measurement-corrected state distribution.
cov	ndarray	covariance matrix of the measurement-corrected state distribution.

ConstantVelocityKalmanFilter

Bases: KalmanFilter

A simple Kalman filter for tracking bounding boxes in image space. The 8-dimensional state space

x, y, a, h, vx, vy, va, vh

contains the bounding box center position (x, y), aspect ratio a, height h, and their respective velocities.

Object motion follows a constant velocity model. The bounding box location (x, y, a, h) is taken as direct observation of the state space (linear observation model).

__init__(*, std_weight_position=1 / 20, std_weight_velocity=1 / 160)

Other Parameters:

Name	Type	Description
std_weight_position	float	Relative observation uncertainty.
std_weight_velocity	float	Relative velocity uncertainty.

multi_predict(mean, covariance)

vectorized version of the prediction step (Vectorized version).

Parameters:

Name	Type	Description	Default
mean		Nx8 dimensional mean matrix of the object states at the previous time step.	required
covariance		Nx8x8 dimensional covariance matrics of the object states at the previous time step.	required

Returns:

Name	Type	Description
mean	ndarray	the mean vector of the predicted state
cov	ndarray	the covariance matrix of the predicted state

gating_distance(mean, covariance, measurements, only_position=False, metric='maha')

Compute gating distance between state distribution and measurements. A suitable distance threshold can be obtained from chi2inv95. If only_position is False, the chi-square distribution has 4 degrees of freedom, otherwise 2.

Parameters:

Name	Type	Description	Default
mean		Mean vector over the state distribution (8 dimensional).	required
covariance		Covariance of the state distribution (8x8 dimensional).	required
measurements		An Nx4 dimensional matrix of N measurements, each in format (x, y, a, h) where (x, y) is the bounding box center position, a the aspect ratio, and h the height.	required
only_position		If True, distance computation is done with respect to the bounding box center position only.	False

Returns:

Name	Type	Description
gating_distance	ndarray	an array of length N, where the i-th element contains the squared Mahalanobis distance between (mean, covariance) and measurements[i].

ConstantVelocityKalmanFilter3D

Bases: KalmanFilter

A 3D Kalman filter for tracking 3D bounding boxes follows a constant velocity model. The 8-dimensional state space are:

z, y, x, d, h, w, vz, vy, vx, vd, vh, vw

__init__(*, std_weight_position=1 / 20, std_weight_velocity=1 / 160, z_scaling=1)

Other Parameters:

Name	Type	Description
std_weight_position	float	Relative observation uncertainty.
std_weight_velocity	float	Relative velocity uncertainty.
z_scaling	float	relative pixel size along z-axis vs x-y-axis

multi_predict(mean, covariance)

Vectorized Kalman filter prediction step.

KTracker

A stateful KalmanFilter tracker

__init__(init_obs, frame_id, kf, *, data={})

constructor

Parameters:

Name	Type	Description	Default
init_obs	ArrayLike	initial measurement space data	required
frame_id	int	the frame number	required
kf	KalmanFilter	a Kalman Filter following the KalmanFilter protocol	required

Other Parameters:

Name	Type	Description
data	dict	additional data of the observation that will be store

predict()

Kalman filter prediction

initialize()

Initialze the track, so that it has a track id and a history object

update(new_track)

Kalman filter update step

new_track: the new observation

mark_lost(*, update_state=None)

Mark the track as lost

Other Parameters:

Name	Type	Description
update_state	ArrayLike	if not None, change the mean vector to this one

mark_removed()

Mark the track as removed

assign(tracks, dets, cost_matrix, threshold) staticmethod

perform MOT assigment

Parameters:

Name	Type	Description	Default
tracks	Sequence[KTracker]	current list of tracks of length M	required
dets	Sequence[KTracker]	list of new observations of length N	required
cost_matrix	ArrayLike	[M x N] cost matrix	required
threshold	float	max cost for linking	required

Returns:

Name	Type	Description
tracked	Sequence[KTracker]	list of tracks that are updated with new observations
remaining_tracks	Sequence[KTracker]	list of tracks that did not find a suitable link
remaining_dets	Sequence[KTracker]	the remaining detections that didn't get linked in.

ZStackTracker dataclass

A special tracker for building 3d segmentation from a z-stack of 2d segmentations. It uses a kalman tracker of 2d boxes that assumes a fixed center location and tracking only the size changes.

Attributes:

Name	Type	Description
score_thresh	float	score threshold for high-confidence detections
nms_iou	float	filter the 2D results with non-max-supression
cost_thresh	float	cost threshold for link assignment
cost_thresh_secondary	float	cost threshold for link assignment of low confidence detections
max_init_area	float	maximum segmentation area to be considered as the first slice of a cell
max_time_lost	int	max consecutive missing slices
std_weight_position	float	kalman filter parameter. relative error for position
std_weight_velocity	float	kalman filter parameter. relative error for velocity
min_z_slices	int	minimal z slices needed to be considered a cell
use_generalized_iou_loss	bool	whether to use generalized iou loss instead of simple iou_loss for linking cost

update(tracks, dets, frame_id)

Update the tracker with one frame

Parameters:

Name	Type	Description	Default
tracks	Sequence[KTracker]	a list of KTacker representing currently tracks cells	required
dets	dict	predictor output in bbox format	required
frame_id	int	current frame number	required

Returns:

Name	Type	Description
tracked_tracks	list[KTracker]	list of tracks that are active
removed_tracks	list[KTracker]	list of tracks that are inactive

finalize(tracks, *, fill_in_missing=True)

Finalize step which compute scores and bboxes, and optionally fill in the missing segmentations. The aggregated score is the mean of the top-k 2d scores, where k is the min_z_slices.

Parameters:

Name	Type	Description	Default
tracks	list[KTracker]	list of tracks, each represent a cell in 3d	required

Other Parameters:

Name	Type	Description
fill_in_missing	bool	whether try to generate segmentations for missing slices

Returns:

Type	Description
list[KTracker]	list of tracks will "score", "bbox" fields.

render_label(tracks, img3d_shape) staticmethod

create 3d label from tracking results

Parameters:

Name	Type	Description	Default
tracks	list[KTracker]	list of tracks	required
img3d_shape	Sequence[int]	tuple of (D, H, W)	required

Returns:

Name	Type	Description
label	ndarray	array of shape (D, H, W)