Evaluation utilities¶

flat_bug.eval_utils.bbox_intersect(b1: ndarray, b2s: ndarray) → ndarray¶

Calculate the intersecting rectangle between two rectangles. The rectangles must be aligned with the axes.

Parameters:

b1 (np.ndarray) – Bounding box 1.
b2s (np.ndarray) – Bounding boxes 2.

Returns:

Intersecting rectangles of shape (n, 4).

Return type:

out (np.ndarray)

flat_bug.eval_utils.bbox_intersect_area(b1: ndarray, b2s: ndarray) → ndarray¶

Calculate the area of the intersecting rectangle between two rectangles. The rectangles must be aligned with the axes.

Parameters:

b1 (np.ndarray) – Bounding box 1.
b2s (np.ndarray) – Bounding boxes 2.

Returns:

Area of the intersecting rectangles of shape (n,).

Return type:

out (np.ndarray)

flat_bug.eval_utils.best_confidence_threshold(y: List[int] | ndarray, iou: List[float] | ndarray, confidence: List[float] | ndarray, n: int = 100) → float¶

Finds the best confidence threshold using bootstrapping and F1 score optimization.

Parameters:

y (Union[List[int], np.ndarray]) – Ground truth binary labels.
iou (Union[List[float], np.ndarray]) – IoU values, non-floats default to 0.
confidence (Union[List[float], np.ndarray]) – Confidence scores for predictions, non-floats default to 0.
n (int, optional) – Number of bootstrap samples. Defaults to 100.

Returns:

The average of the optimal thresholds found for each bootstrap sample.

Return type:

out (float)

flat_bug.eval_utils.compare_groups(group1: list, group2: list, threshold: float = 0.1, group_labels: str | None = ['Ground Truth', 'Predictions'], plot: bool = False, plot_scale: float = 1, plot_boxes: bool = True, image_path: str | None = None, output_identifier: str | None = None, output_directory: str | None = None) → str | Dict¶

Compares group 1 to group 2.

Output is saved to a CSV file with the following columns:

Idx_1 (int) is the index of the geometry in group 1
Idx_2 (int) is the index of the matched geometry in group 2, or -1 if there is no match
IoU (float) is the intersection over union, or 0 if there is no match
contourArea_1 (int) is the area of the contour in group 1
contourArea_2 (int) is the area of the contour in group 2, or 0 if there is no match
bbox_1 (list[int, int, int, int : xmin, ymin, xmax, ymax]) is the bounding box of the geometry in group 1
bbox_2 (list[int, int, int, int : xmin, ymin, xmax, ymax]) is the bounding box of the matched geometry in group 2, or an empty list if there is no match
contour_1 (list[list[int, int : x_i, y_i]]) is the contour of the geometry in group 1
contour_2 (list[list[int, int : x_i, y_i]]) is the contour of the matched geometry in group 2, or an empty list if there is no match

Parameters:

group1 (list) – Group 1.
group2 (list) – Group 2.
threshold (float, optional) – IoU threshold for matching elements between groups. Defaults to 0.1.
group_labels (Optional[str], optional) – Group labels. Defaults to [“Ground Truth”, “Predictions”].
plot (bool, optional) – Whether to plot the matches and the IoU matrix, usually this is much slower than simply comparing the groups. Defaults to False.
plot_scale (float, optional) – Scale of the plot. Defaults to 1. Lower values will make the plot smaller, but may be faster.
plot_boxes (bool, optional) – Whether to plot the bounding boxes. Defaults to True.
image_path (Optional[str], optional) – Path to the image. Defaults to None.
output_identifier (Optional[str], optional) – Output identifier. Defaults to None.
output_directory (Optional[str], optional) – Output directory. Defaults to None.

Returns:

Path to the CSV file or the data that would have been saved to the CSV file as a dictionary, where the keys are the column names and the values are the column values.

Return type:

out (Union[str, dict])

flat_bug.eval_utils.contour_intersection(contour1: ndarray, contour2: ndarray, box1: ndarray, box2: ndarray) → ndarray¶

Calculates the intersection of two contours.

Contours should be providedd as [x1, y1, x2, y2, …, xn, yn]

Algorithm: 1. Format the contours to OpenCV format. 2. Calculate the area of the contours. 3. If the area of either contour is 0, return 0. 4. Offset the contours such that the minimum x and y coordinates are 0. 5. Create a mask for each contour, the masks have size (max_x, max_y). 6. Calculate the intersection of the masks by multiplying them element-wise. 7. Return the sum of the intersection mask.

Parameters:

contour1 (np.ndarray) – Contour 1.
contour2 (np.ndarray) – Contour 2.
box1 (np.ndarray) – Bounding box 1.
box2 (np.ndarray) – Bounding box 2.

Returns:

Scalar intersection of the contours with type np.int64.

Return type:

out (np.ndarray)

flat_bug.eval_utils.equal_spaced_cuts(k: int, start: float | int, end: float | int) → ndarray¶

Generate k equal spaced cuts between start and end.

The edges are not included, and the distance between the left-most and right-most cut to the edges is half the distance between the cuts.

Parameters:

k (int) – Number of cuts.
start (float) – Start value.
end (float) – End value.

Returns:

Cuts.

Return type:

out (np.ndarray)

flat_bug.eval_utils.f1_score(GT: ndarray, MP: ndarray) → float¶

Calculates the F1 score for a binary classification problem.

Parameters:

GT (np.ndarray) – Ground truth binary labels.
MP (np.ndarray) – Predicted binary labels.

Returns:

The F1 score.

Return type:

out (float)

flat_bug.eval_utils.format_cell(cell: str, digits: int = 3, max_length: int = 30) → str¶

Autoformat a cell for a table.

Standardizes the number of decimals if the cell is coercible to a float, and truncates the cell if it exceeds the maximum length.

Parameters:

cell (str) – The cell to format.
digits (int, optional) – Number of digits to display for floats. Default is 3.
max_length (int, optional) – Maximum number of characters in the output string. Default is 30. OBS: Paths are not truncated.

Returns:

The formatted cell string where length <= max_length.

Return type:

out (str)

flat_bug.eval_utils.format_row(cells: List[Any], widths: List[int], align: str = 'center') → str¶

Format a row of a table.

Parameters:

cells (List[Any]) – The cells of the row. Elements should be compatible with f-strings (__format__).
widths (List[int]) – The widths of each column.
align (str, optional) – Alignment of the cell content within each column. Valid options are “center”/”left”/”right”. Defaults to “center”.

Returns:

The formatted row.

Return type:

out (str)

flat_bug.eval_utils.generate_block(min: int, max: int, size: int) → ndarray¶

Generates a block of integers centered around a random start value within a given range.

Parameters:

min (int) – Minimum value for the block.
max (int) – Maximum value for the block.
size (int) – Size of the block to generate.

Returns:

Array of integers within the specified range.

Return type:

out (np.ndarray)

flat_bug.eval_utils.generate_bootstraps(s: int, n: int, block: bool = False) → List[ndarray]¶

Generates bootstrap samples with or without block sampling.

Parameters:

s (int) – The size of the dataset.
n (int) – The number of bootstrap samples to generate.
block (bool, optional) – If True, generates block-based bootstraps. Defaults to False.

Returns:

List of bootstrap samples.

Return type:

out (List[np.ndarray])

flat_bug.eval_utils.match_geoms(contours1: List[ndarray], contours2: List[ndarray], threshold: float = 0.25, iou_mat: ndarray | None = None, areas1: ndarray | None = None, areas2: ndarray | None = None) → Tuple[ndarray, int]¶

Matches geometries (polygons) in group 1 to geometries in group 2.

Parameters:

contours1 (List[np.ndarray]) – Geometries (polygons) in group 1. List of length N, where each element is a Xx2 array of contour coordinates.
contours2 (List[np.ndarray]) – Geometries (polygons) in group 2. List of length M, where each element is a Xx2 array of contour coordinates.
threshold (float, optional) – IoU threshold. Defaults to 1/4.
iou_mat (Optional[np.ndarray], optional) – IoU matrix of size NxM. Computed if None. Defaults to None.
areas1 (Optional[np.ndarray], optional) – Areas of polygons in group 1. Computed if None. Defaults to None.
areas2 (Optional[np.ndarray], optional) – Areas of polygons in group 2. Computed if None. Defaults to None.

Returns:

Nx2 array of matched indices from group 1 and group 2, and the number of unmatched geometries in group 2.

Return type:

out (List[np.ndarray, int])

flat_bug.eval_utils.optimal_threshold_f1(y: ndarray, iou: ndarray, confidence: ndarray, num_thresholds: int = 100) → float¶

Finds the optimal threshold for F1 score by iterating over possible thresholds.

Parameters:

y (np.ndarray) – Ground truth binary labels.
iou (np.ndarray) – IoU values,
confidence (np.ndarray) – Confidence scores for predictions.
num_thresholds (int, optional) – Number of thresholds to test. Defaults to 100.

Returns:

The threshold that maximizes the F1 score.

Return type:

out (float)

flat_bug.eval_utils.pairwise_contour_intersection(contours1: List[ndarray], contours2: List[ndarray] | None = None, bboxes1: ndarray | None = None, bboxes2: ndarray | None = None, areas1: ndarray | None = None, areas2: ndarray | None = None) → array¶

Calculates the pairwise intersection of two groups of contours.

Parameters:

contours1 (List[np.ndarray]) – Contours in group 1.
contours2 (Optional[np.ndarray]) – Contours in group 2. If None provided, symmetric intersection is calculated for contours1 instead. Defaults to None.
areas1 (Optional[np.ndarray]) – Areas of contours in group 1. Computed if not None. Defaults to None
areas2 (Optional[np.ndarray]) – Areas of contours in group 2. Computed if not None. Defaults to None
bboxes1 (Optional[np.ndarray]) – Bounding boxes of contours in group 1. Computed if not None. Defaults to None
bboxes2 (Optional[np.ndarray]) – Bounding boxes of contours in group 2. Computed if not None. Defaults to None

Returns:

Intersection matrix of shape (n, m).

Return type:

out (np.ndarray)

flat_bug.eval_utils.plot_heatmap(mat: ndarray, axis_labels: List[str] | None = None, breaks: int = 25, dimensions: Tuple[int, int] | None = None, output_path: str = None, scale: float = 1)¶

Plots a heatmap of a matrix using OpenCV.

Parameters:

mat (np.ndarray) – Matrix to plot.
axis_labels (Optional[List[str]], optional) – Axis labels. Defaults to None.
breaks (int, optional) – Number of breaks on the colorbar. Defaults to 25.
dimensions (Optional[Tuple[int, int]], optional) – Dimensions of the output image. Defaults to None.
output_path (str, optional) – Output path. Defaults to None.
scale (float, optional) – Scale of the output image. Defaults to 1.

flat_bug.eval_utils.plot_matches(matches: ndarray, contours1: list[ndarray], contours2: List[ndarray], group_labels: List[str] | None = None, image_path: str | None = None, output_path: str | None = None, scale: float = 1, boxes: bool = True)¶

Plots the matches between two groups of contours using OpenCV.

Parameters:

matches (np.ndarray) – Matches between group 1 and 2.
contours1 (List[np.ndarray]) – Contours of group 1.
contours2 (List[np.ndarray]) – Contours of group 2.
group_labels (Optional[List[str]], optional) – Labels of the two groups (should have a length of 2). If None the groups are labelled as “1” and “2”. Defaults to None.
image_path (Optional[str], optional) – Path to the image. If None the matches are plotted on a black background. Defaults to None.
output_path (Optional[str], optional) – Output path of plot. If None the rasterized result is displayed or returned as an array, depending in the context. Defaults to None.
scale (float, optional) – Scale of the output image. Defaults to 1.
boxes (bool, optional) – Flag to indicate whether to plot bounding boxes. Defaults to True.

flat_bug.eval_utils.pretty_print_csv(csv_file: str, delimiter: str = ',')¶

Pretty print the CSV file.

Parameters:: csv_file (str) – The path to the CSV file.