lacss.tracking
KalmanFilter
Bases: Protocol
KalmanFilter Protocol definition
initialize(measurement)
Create track from unassociated measurement.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
measurement |
data in measurement space |
required |
Returns:
Name | Type | Description |
---|---|---|
mean |
ndarray
|
Unobserved velocities are initialized to 0 mean. |
cov |
ndarray
|
covariance matrix (8x8 dimensional) of the new track. |
predict(mean, covariance)
Run Kalman filter prediction step.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
mean |
mean vector of the object state at the previous time step. |
required | |
covariance |
The covariance matrix of the object state at the previous time step. |
required |
Returns:
Name | Type | Description |
---|---|---|
mean |
ndarray
|
the mean vector of the predicted state |
cov |
ndarray
|
the covariance matrix of the predicted |
update(mean, covariance, measurement)
Run Kalman filter correction step.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
mean |
The predicted state's mean vector. |
required | |
covariance |
The state's covariance matrix. |
required | |
measurement |
The measurement vector |
required |
Returns:
Name | Type | Description |
---|---|---|
mean |
ndarray
|
mean vector of the measurement-corrected state distribution. |
cov |
ndarray
|
covariance matrix of the measurement-corrected state distribution. |
ConstantVelocityKalmanFilter
Bases: KalmanFilter
A simple Kalman filter for tracking bounding boxes in image space. The 8-dimensional state space
x, y, a, h, vx, vy, va, vh
contains the bounding box center position (x, y), aspect ratio a, height h, and their respective velocities.
Object motion follows a constant velocity model. The bounding box location (x, y, a, h) is taken as direct observation of the state space (linear observation model).
__init__(*, std_weight_position=1 / 20, std_weight_velocity=1 / 160)
Other Parameters:
Name | Type | Description |
---|---|---|
std_weight_position |
float
|
Relative observation uncertainty. |
std_weight_velocity |
float
|
Relative velocity uncertainty. |
multi_predict(mean, covariance)
vectorized version of the prediction step (Vectorized version).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
mean |
Nx8 dimensional mean matrix of the object states at the previous time step. |
required | |
covariance |
Nx8x8 dimensional covariance matrics of the object states at the previous time step. |
required |
Returns:
Name | Type | Description |
---|---|---|
mean |
ndarray
|
the mean vector of the predicted state |
cov |
ndarray
|
the covariance matrix of the predicted state |
gating_distance(mean, covariance, measurements, only_position=False, metric='maha')
Compute gating distance between state distribution and measurements.
A suitable distance threshold can be obtained from chi2inv95
. If
only_position
is False, the chi-square distribution has 4 degrees of
freedom, otherwise 2.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
mean |
Mean vector over the state distribution (8 dimensional). |
required | |
covariance |
Covariance of the state distribution (8x8 dimensional). |
required | |
measurements |
An Nx4 dimensional matrix of N measurements, each in format (x, y, a, h) where (x, y) is the bounding box center position, a the aspect ratio, and h the height. |
required | |
only_position |
If True, distance computation is done with respect to the bounding box center position only. |
False
|
Returns:
Name | Type | Description |
---|---|---|
gating_distance |
ndarray
|
an array of length N, where the i-th element contains the
squared Mahalanobis distance between (mean, covariance) and |
ConstantVelocityKalmanFilter3D
Bases: KalmanFilter
A 3D Kalman filter for tracking 3D bounding boxes follows a constant velocity model. The 8-dimensional state space are:
z, y, x, d, h, w, vz, vy, vx, vd, vh, vw
__init__(*, std_weight_position=1 / 20, std_weight_velocity=1 / 160, z_scaling=1)
Other Parameters:
Name | Type | Description |
---|---|---|
std_weight_position |
float
|
Relative observation uncertainty. |
std_weight_velocity |
float
|
Relative velocity uncertainty. |
z_scaling |
float
|
relative pixel size along z-axis vs x-y-axis |
multi_predict(mean, covariance)
Vectorized Kalman filter prediction step.
KTracker
A stateful KalmanFilter tracker
__init__(init_obs, frame_id, kf, *, data={})
constructor
Parameters:
Name | Type | Description | Default |
---|---|---|---|
init_obs |
ArrayLike
|
initial measurement space data |
required |
frame_id |
int
|
the frame number |
required |
kf |
KalmanFilter
|
a Kalman Filter following the KalmanFilter protocol |
required |
Other Parameters:
Name | Type | Description |
---|---|---|
data |
dict
|
additional data of the observation that will be store |
predict()
Kalman filter prediction
initialize()
Initialze the track, so that it has a track id and a history object
update(new_track)
Kalman filter update step
new_track: the new observation
mark_lost(*, update_state=None)
Mark the track as lost
Other Parameters:
Name | Type | Description |
---|---|---|
update_state |
ArrayLike
|
if not None, change the mean vector to this one |
mark_removed()
Mark the track as removed
assign(tracks, dets, cost_matrix, threshold)
staticmethod
perform MOT assigment
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tracks |
Sequence[KTracker]
|
current list of tracks of length M |
required |
dets |
Sequence[KTracker]
|
list of new observations of length N |
required |
cost_matrix |
ArrayLike
|
[M x N] cost matrix |
required |
threshold |
float
|
max cost for linking |
required |
Returns:
Name | Type | Description |
---|---|---|
tracked |
Sequence[KTracker]
|
list of tracks that are updated with new observations |
remaining_tracks |
Sequence[KTracker]
|
list of tracks that did not find a suitable link |
remaining_dets |
Sequence[KTracker]
|
the remaining detections that didn't get linked in. |
ZStackTracker
dataclass
A special tracker for building 3d segmentation from a z-stack of 2d segmentations. It uses a kalman tracker of 2d boxes that assumes a fixed center location and tracking only the size changes.
Attributes:
Name | Type | Description |
---|---|---|
score_thresh |
float
|
score threshold for high-confidence detections |
nms_iou |
float
|
filter the 2D results with non-max-supression |
cost_thresh |
float
|
cost threshold for link assignment |
cost_thresh_secondary |
float
|
cost threshold for link assignment of low confidence detections |
max_init_area |
float
|
maximum segmentation area to be considered as the first slice of a cell |
max_time_lost |
int
|
max consecutive missing slices |
std_weight_position |
float
|
kalman filter parameter. relative error for position |
std_weight_velocity |
float
|
kalman filter parameter. relative error for velocity |
min_z_slices |
int
|
minimal z slices needed to be considered a cell |
use_generalized_iou_loss |
bool
|
whether to use generalized iou loss instead of simple iou_loss for linking cost |
update(tracks, dets, frame_id)
Update the tracker with one frame
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tracks |
Sequence[KTracker]
|
a list of KTacker representing currently tracks cells |
required |
dets |
dict
|
predictor output in bbox format |
required |
frame_id |
int
|
current frame number |
required |
Returns:
Name | Type | Description |
---|---|---|
tracked_tracks |
list[KTracker]
|
list of tracks that are active |
removed_tracks |
list[KTracker]
|
list of tracks that are inactive |
finalize(tracks, *, fill_in_missing=True)
Finalize step which compute scores and bboxes, and optionally fill in the missing segmentations. The aggregated score is the mean of the top-k 2d scores, where k is the min_z_slices.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tracks |
list[KTracker]
|
list of tracks, each represent a cell in 3d |
required |
Other Parameters:
Name | Type | Description |
---|---|---|
fill_in_missing |
bool
|
whether try to generate segmentations for missing slices |
Returns:
Type | Description |
---|---|
list[KTracker]
|
list of tracks will "score", "bbox" fields. |
render_label(tracks, img3d_shape)
staticmethod
create 3d label from tracking results
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tracks |
list[KTracker]
|
list of tracks |
required |
img3d_shape |
Sequence[int]
|
tuple of (D, H, W) |
required |
Returns:
Name | Type | Description |
---|---|---|
label |
ndarray
|
array of shape (D, H, W) |