Package biopb.image
Services
service ObjectDetection
Unitary call for computing cell detection / segmentation.
rpc RunDetectionStream
Request: biopb.image.DetectionRequest
Response:
biopb.image.DetectionResponse
The streaming version allows dynamic updating of the DetectionRequest and getting updated results. For example, a client may request analyses at several different settings on the same image. After initial request, the following streaming requests no long need to transmit the image pixel data anymore, but only the new parameter settings, which saves network bandwidth.
rpc RunDetectionOnGrid
Request: biopb.image.DetectionRequest
Response:
biopb.image.DetectionResponse
Send the image dataset as a stream of (overlapping or non-overlapping) image patches, instead of one large chunk of binary data. There are two use cases for this call: (1) The image dataset is too large therefore sending all data in one message would exceed GRPC message size limit; (2) The original data were taken as a grid scan so it is convinent to read/write as patches.
Run model adaptation based on the submitted stream of DetectionRequest, which may or may not contain ROI annotations. The all returns an opaque data strcuture representing the model configuration after model adaptation, which in turn should be included in future inference reuest as a DetectionSetting field.
service ProcessImage
A generic image process protocol that returns a new image. Examples include denoising, super-resolution, and semantic segmentation.
Process the input image
Query operation names. Server with a single op may not implement this call.
Messages
message BinData
A binary stream
Field | Type | Description | |
---|---|---|---|
1 | data |
bytes
|
|
2 | endianness |
biopb.image.BinData.Endianness
|
message DetectionRequest
Field | Type | Description | |
---|---|---|---|
1 | image_data |
biopb.image.ImageData
|
Encapsulate all pixel data and associated metadata. In a streaming call, this field is allowed to be null, which simply is a request to re-analyze the last submitted image with the updated settings. If this is the first call in a stream, the server will not return results, but simply update and remember the settings. |
2 | detection_settings |
biopb.image.DetectionSettings
|
Other parameters associated with the request, such as score threshold etc. In a streaming call, this field is allowed to be null, in which case the server will use the setings of the previous request. |
3 | field_mask |
google.protobuf.FieldMask
|
A field_mask to indicate the requested fields in the reply. Default is to return the full response. |
message DetectionSettings
Field | Type | Description | |
---|---|---|---|
1 | min_cell_area |
optional
float
|
Minimal cell area to be consider for detection. The unit is decided by
the physical unit of the image, i.e., if the physical pixel size unit is
"nm", then the unit here is nm^2 (2D) or nm^3 (3D). However, if the
physical pixel size field is not set, then the unit here is |
2 | min_score |
optional
float
|
Minimal detection score in order to be considered. If unset, the default value is chosen server-side. |
3 | segmentation_threshold |
optional
float
|
Threshold for binarize segmentation logits. If unset, default to 0. |
4 | nms_iou |
optional
float
|
A threshold value for remove overlapping detections. If two detection has an IOU > this threshold, the one with the lower score will be eliminated from the detection list. Nothing will be removed if the field is not set. |
8 | model_name |
string
|
Specify the model name for servers that support more than one model variants. |
9 | model_configuration |
google.protobuf.Any
|
An opaque data structure that specify a server side model configuration. An important use case is to perform few-shot model adaptations. The model adaptation call returns a new model configuration. This stucture should be included in all new inference requests, so that the server understand that it should perform computation using the adapted model. |
oneof resize_info
|
|||
6 | cell_diameter_hint |
float
|
A hint for the typical cell diameter. If the physical pixel size is known, the value takes the same physical unit. Otherwise, the unit is pixel. |
7 | scaling_hint |
float
|
A direct suggestion for image scaling. A nonnegative value. However, requesting a very large scaling factor will likely be refused by the server due to the potential out-of-memory problem. |
message Ellipse
A simple ellipse object.
Field | Type | Description | |
---|---|---|---|
1 | center |
biopb.image.Point
|
|
2 | radius |
biopb.image.Point
|
If the |
message Face (Nested in biopb.image.Mesh )
A trangle defined three indices of the verts array
Field | Type | Description | |
---|---|---|---|
1 | p1 |
uint32
|
|
2 | p2 |
uint32
|
|
3 | p3 |
uint32
|
|
4 | normals |
repeated
biopb.image.Point
|
Optionally one can cache the computed normal vectors here |
5 | values |
repeated
float
|
Optionally one can cache the associated face values here |
message ImageAnnotation
Additional annotation of the image, which may be used for the analysis.
Field | Type | Description | |
---|---|---|---|
1 | rois |
repeated
biopb.image.ROI
|
a list of region of interest |
2 | annotation |
string
|
additional text annotation |
message ImageData
Field | Type | Description | |
---|---|---|---|
1 | pixels |
biopb.image.Pixels
|
Encapsulated image data |
2 | image_annotation |
biopb.image.ImageAnnotation
|
Optional additional information regarding the image that will be utilized by the RPC call. For example, some SegmentAnything type models may ask the user to provide a tentative bounding box for the objects that the users are interested in. |
message Mask
The Mask ROI shape is a link to a BinData object that is a BIT mask drawn on top of the image as an ROI. It is applied at the same scale, pixel to pixel, as the Image the ROI is applied to. The dimension order of the mask is the same as the that of the image data the Roi is applied to.
Field | Type | Description | |
---|---|---|---|
1 | rectangle |
biopb.image.Rectangle
|
|
2 | bin_data |
biopb.image.BinData
|
message Mesh
A closed 3D shape repesented by triangular meshes.
Field | Type | Description | |
---|---|---|---|
1 | verts |
repeated
biopb.image.Point
|
All vertices of the mesh |
2 | faces |
repeated
biopb.image.Mesh.Face
|
message Pixels
The Pixels element represent generic 5D pixel data. The time axis is currently not being utilized, but is defined here for futire extensibility. This is modeled after OME-XML's Pixel structure.
Field | Type | Description | |
---|---|---|---|
3 | bindata |
biopb.image.BinData
|
Pixel values |
4 | dimension_order |
string
|
A string describing the dimension order of the pixel data, e.g. "XYZCT" The string must be a permutaion of these 5 letters even if some of the dimension doesn't exist in the data. |
5 | dtype |
string
|
A string describing the data type using numpy's convention. E.g., 'i8' represent int64. Endianness indicator, e.g. '=i8', is allowed but will be ignored. Instead the endianness will be determined by the field within the Bindata structure. |
9 | size_x |
uint32
|
Dimensional size of pixel data array [units:none] |
10 | size_y |
uint32
|
|
11 | size_z |
uint32
|
|
12 | size_c |
uint32
|
|
13 | size_t |
uint32
|
|
14 | physical_size_x |
optional
float
|
Physical size of a pixel. |
15 | physical_size_y |
optional
float
|
|
16 | physical_size_z |
optional
float
|
|
17 | physical_size_unit |
string
|
physical size unit, default "µm" |
18 | time_increment |
optional
float
|
time_increment is used for time series that have a global timing specification instead of per-timepoint timing info. For example in a video stream. Units are set by TimeIncrementUnit. |
21 | time_increment_unit |
string
|
time unit, default "s" |
22 | offset_x |
optional
uint32
|
for large image dataset, it is necessary to send a stream of image patches instead of one monolithic data structure of the whole dataset. The offset values allows one to specify the relative locations of each patch. |
23 | offset_y |
optional
uint32
|
|
24 | offset_z |
optional
uint32
|
|
25 | offset_t |
optional
uint32
|
message Point
/////////// ROI classes are desgined roughly after OME-XML's related datatypes, but with a emphasis on 3D data
Field | Type | Description | |
---|---|---|---|
1 | x |
float
|
|
2 | y |
float
|
|
3 | z |
optional
float
|
message Polygon
The Polygon defines closed shapes formed of straight lines. Note: Polygon uses counterclockwise winding (this is the default OpenGL behavior)
Field | Type | Description | |
---|---|---|---|
1 | points |
repeated
biopb.image.Point
|
message ProcessRequest
Field | Type | Description | |
---|---|---|---|
1 | image_data |
biopb.image.ImageData
|
Input image for processing |
2 | op_name |
string
|
Specify a operator name because the server may implement multiple operations. |
3 | field_mask |
google.protobuf.FieldMask
|
A field_mask to indicate the requested fields in the reply. Default is to return the full response. May be ignored by the server. |
message ProcessResponse
Field | Type | Description | |
---|---|---|---|
1 | image_data |
biopb.image.ImageData
|
Output image after processing |
2 | annotation |
string
|
Additonal information the server may return as a text string |
message ROI
A three dimensional 'Region of Interest'.
Field | Type | Description | |
---|---|---|---|
oneof shape
|
|||
1 | point |
biopb.image.Point
|
|
2 | rectangle |
biopb.image.Rectangle
|
|
3 | ellipse |
biopb.image.Ellipse
|
|
4 | polygon |
biopb.image.Polygon
|
|
5 | mesh |
biopb.image.Mesh
|
|
6 | mask |
biopb.image.Mask
|
message Rectangle
A simple rectangle object.
Field | Type | Description | |
---|---|---|---|
1 | top_left |
biopb.image.Point
|
|
2 | bottom_right |
biopb.image.Point
|
Enums
enum Endianness
Data endianness. Note OME-XML use a bool. We change that to a Enum that default to BigEndian for slightly better efficiency and allowing potential extension to other states: e.g. system default.
Name | Number | Description |
---|---|---|
BIG |
0 | |
LITTLE |
1 |