lacss.ops

Ops on bounding-boxes

All functions here are degisned to work as either a numpy op or a jax op depending on the data type of the input.

box_area(box)

Computes area of boxes. Args: box: a float Tensor with [..., N, 4].

Returns:

Type	Description
ArrayLike	a float Tensor with [..., N]

box_intersection(gt_boxes, boxes)

Compute pairwise intersection areas between boxes.

Parameters:

Name	Type	Description	Default
gt_boxes	ArrayLike	[..., N, 4]	required
boxes	ArrayLike	[..., M, 4]	required

Returns:

Type	Description
ArrayLike	a float Tensor with shape [..., N, M] representing pairwise intersections.

box_iou_similarity(gt_boxes, boxes)

Computes pairwise intersection-over-union between box collections.

Parameters:

Name	Type	Description	Default
gt_boxes	ArrayLike	a float Tensor with [..., N, 4].	required
boxes	ArrayLike	a float Tensor with [..., M, 4].	required

Returns:

Type	Description
ArrayLike	a Tensor with shape [..., N, M] representing pairwise iou scores.

sorbel_edges(image)

Returns a tensor holding Sobel edge maps.

Examples:

>>> image = random.uniform(key, shape=[3, 28, 28])
>>> sobel = sobel_edges(image)
>>> sobel_y = sobel[0, :, :, :] # sobel in y-direction
>>> sobel_x = sobel[1, :, :, :] # sobel in x-direction

Parameters:

Name	Type	Description	Default
image	ArrayLike	[n, h, w]	required

Returns:

Type	Description
Array	Tensor holding edge maps for each channel. [2, n, h, w]

sub_pixel_crop_and_resize(img, bbox, output_shape, out_of_bound_value=0)

Retrieve image values of a bbox. Resize output to output_shape. Used for ROI-Align.

Parameters:

Name	Type	Description	Default
img	ArrayLike	Array of shape [H, W, ...]	required
bbox	ArrayLike	[y0, x0, y1, x1]	required
output_shape	tuple[int]	[h, w]	required
out_of_bound_value	float	optional float constant, defualt 0.	0

Returns:

Name	Type	Description
values	Array	[h, w, ...], float

sub_pixel_samples(img, locs, out_of_bound_value=0, edge_indexing=False)

Retrieve image values as non-integer locations by interpolation

Parameters:

Name	Type	Description	Default
img	ArrayLike	Array of shape [D1,D2,..,Dk, ...]	required
locs	ArrayLike	Array of shape [d1,d2,..,dn, k]	required
out_of_bound_value	float	optional float constant, defualt 0.	0
edge_indexing	bool	if True, the index for the top/left pixel is 0.5, otherwise 0. Default is False	False

Returns:

Name	Type	Description
values	Array	[d1,d2,..,dn, ...], float

distance_similarity(pred_locations, gt_locations)

Compute distance similarity matrix

pairwise similarity = 1 / distance ^2

Parameters:

Name	Type	Description	Default
pred_locations	ArrayLike	[N, 2] use -1 to mask out invalid locations	required
gt_locations	ArrayLike	[K, 2] use -1 to mask out invalid locations	required

Returns:

Name	Type	Description
similarity_matrix	Array	[N, k]

location_matching(pred_locations, gt_locations, threshold)

Match predicted location to gt locations

Parameters:

Name	Type	Description	Default
pred_locations	ArrayLike	r [N, 2]	required
gt_locations	ArrayLike	[K, 2]	required
threshold	float	float. Maximum distance to be matched	required

Returns:

Name	Type	Description
matches	Array	[N], indices of the matches location in gt list
indicators	Array	[N] bool

locations_to_labels(locations, target_shape, threshold=1.5)

Generate labels as LPN regression targets

Parameters:

Name	Type	Description	Default
locations	ArrayLike	[N, 2] float32 true location values. scaled 0..1, masking out invalid with -1	required
target_shape	Shape	(H, W) int	required
threshold	float	distance threshold for postive label	1.5

Returns:

Name	Type	Description
score_target	Array	[H, W, 1] int32
regression_target	Array	[H, W, 2] float tensor

sorted_non_max_suppression(scores, boxes, max_output_size, threshold=0.5, min_score=0, return_selection=False)

non-maximum suppression for either bboxes or points.

Assumption:

* The boxes are sorted by scores

The overal design of the algorithm is to handle boxes tile-by-tile:

Parameters:

Name	Type	Description	Default
scores	ArrayLike	[N]	required
boxes	ArrayLike	[N, C] C=4 for boxes, C=2 for locations	required
max_output_size	int	a positive scalar integer	required
threshold	float	a scalar float, can be negative	0.5
min_score	float	min score to be selected, default 0	0
return_selection	bool	whether also return the boolean indicator	False

Returns:

Name	Type	Description
nms_scores	tuple[Array]	[M]. M = max_output_size
nms_proposals	tuple[Array]	[M, C].
selection	tuple[Array]	[N] a boolean indicator of selection status of original input

Various functions deals with segmentation pathces

All functions here takes unbatched input. Use vmap to convert to batched data

bboxes_of_patches(pred, threshold=0.5)

Compute the instance bboxes from model predictions

Parameters:

Name	Type	Description	Default
pred	Sequence | DataDict	A model prediction dictionary:	required
threshold	float	for segmentation, default 0.5	0.5

Returns:

Name	Type	Description
bboxes	ndarray	[n, 4] bboox for empty patches are filled with -1

patches_to_label(pred, input_size, mask=None, score_threshold=0.5, threshold=0.5)

convert patch output to the image label

Parameters:

Name	Type	Description	Default
pred	DataDict	A model prediction dictionary	required
input_size	Shape	shape of the input image. Tuple of H, W	required
mask	Optional[ArrayLike]	boolean indicators masking out unwanted instances. Default is None (all cells)	None
score_threshold	float	otional, min_score to be included. Default is .5.	0.5
threshold	float	segmentation threshold. Default .5	0.5

Returns:

Name	Type	Description
label	Array	[height, width]

patches_to_segmentations(pred, input_size, threshold=0.5)

Expand the predicted patches to the full image size. The default model segmentation output shows only a small patch around each instance. This function expand each patch to the size of the orginal image.

Parameters:

Name	Type	Description	Default
pred	DataDict	A model prediction dictionary	required
input_size	Shape	shape of the input image. Tuple of H, W	required
threshold	float	for segmentation. Default is 0.5.	0.5

Returns:

Name	Type	Description
segmentations	Array	[n, height, width] n full0-size segmenatation masks.

rescale_patches(pred, scale)

Rescale/resize instance outputs in a sub-pixel accurate way. If the input image was rescaled, this function take care of rescaling the predictions to the orginal coodinates.

Parameters:

Name	Type	Description	Default
pred	DataDict	A model prediction dictionary	required
scale	float	The scaling value. The function does not take a noop shortcut even if scale is 1.	required

A tuple of three arrays

Name	Type	Description
patches	Array	a 3D array of the rescaled segmentation patches. The array shape should be different from the orignal patches in model predition.
yy	Array	The y-coordinates of the patches in mesh-grid format
xx	Array	The x-coordinates of the patches in mesh-grid format