Cheat sheets

Detection and Segmentation Compared

Boxes, pixel masks, instance masks, keypoints and text-prompted masks. Outputs, losses, metrics and fixes side by side.

  • Concepts and terms
  • Tasks and commands
  • Annotated code
  • Diagrams
  • Comparisons
  • Defaults and numbers
  • Troubleshooting
  • Questions and tips

Source

Side by side#

TaskOutputMetric
Detection (YOLO)(N, H, W, num_anchors, 5 + C)mAP@0.5:0.95, AP@0.5
Semantic (U-Net)(N, C, H, W)IoU per class, mIoU, Dice
Instance (Mask R-CNN)(N, 1, H, W)Mask AP and box mAP
Pose (heatmaps)(N, K, H, W)OKS, PCK
Open vocab (SAM 3)boxes + labels + scores + masks + IDsMask IoU

Use X when#

YOLO-style detector
boxes and classes in one forward pass, in real time.
U-Net semantic
one class per pixel. Two touching cars become one blob.
Mask R-CNN instance
one mask per object, also when objects share a class.
Panoptic
every pixel gets a class. Every instance gets a unique id.
Top-down pose
highest accuracy. Cost grows with the number of people.
Bottom-up pose
crowds. Constant time for any crowd size.
SAM 3
masks and instance IDs for a noun phrase in one forward pass.
Grounded SAM 2 or YOLO-World
a detector swap or other license. YOLO-World for boxes at high fps.

Mask R-CNN#

top-K proposals7x714x14NMSclientInputimage tensormodelResNet backbonestrides 4, 8, 16, 32FPNtop-down + lateralRPN~1000 proposals, NMS0.7RoIAlignbilinear, no roundingBox headclass + refineMask head28x28 per classOutputboxes + classes +maskstop-K proposals7x714x14NMSclientInputimage tensormodelResNet backbonestrides 4, 8, 16,32FPNtop-down +lateralRPN~1000 proposals,NMS 0.7RoIAlignbilinear, noroundingBox headclass + refineMask head28x28 per classOutputboxes + classes +masks

YOLO inference: postprocess#

1Headone 1x1 conv gives (N, H, W, num_anchors, 5 +C).2Unpackeach anchor holds tx, ty, tw, th, obj and Cclass logits.3Scoresigmoid(obj) times the best class sigmoid.4Thresholddrop the box when the score is belowconf_threshold.5Decodecentre is sigmoid offset plus cell, timesstride. Size is anchor times np.exp.6NMSnms(boxes, scores, iou_threshold) keeps one boxper object.7Returnboxes[keep], scores[keep], classes[keep].

Formulas#

YOLO output
S \times S \times B(5 + C). VOC with S=13, B=2, C=20 gives 50 per cell.
Decode
c_x = (\sigma(t_x) + \mathrm{cell}_x) \cdot \mathrm{stride}, w = a_w e^{t_w}
IoU
area of the intersection over area of the union. A hit needs IoU >= 0.5.
mAP@0.5:0.95
mean AP over IoU 0.5, 0.55, ..., 0.95. The COCO metric.
Dice
2 \sum p y / (\sum p + \sum y + \epsilon). The loss is 1 - Dice.
Dice from IoU
\mathrm{Dice} = 2 \cdot \mathrm{IoU} / (1 + \mathrm{IoU})
Mask R-CNN loss
rpn_cls + rpn_box + box_cls + box_reg + mask.
Heatmap target
\exp(-((x - c_x)^2 + (y - c_y)^2) / 2\sigma^2)

Non-max suppression#

def nms(boxes, scores, iou_threshold=0.45):
order = np.argsort(-scores)best score first
keep = []
while len(order) > 0:
i = order[0]
keep.append(i)
if len(order) == 1:
break
rest = order[1:]
ious = box_iou(boxes[[i]], boxes[rest])[0]
order = rest[ious <= iou_threshold]drop the overlaps
return np.array(keep, dtype=np.int64)

U-Net losses#

def dice_loss(logits, targets, num_classes, eps=1e-6):
probs = F.softmax(logits, dim=1)
targets_one_hot = F.one_hot(targets, num_classes).permute(0, 3, 1, 2).float()
dims = (0, 2, 3)batch, H, W
intersection = (probs * targets_one_hot).sum(dim=dims)
denom = probs.sum(dim=dims) + targets_one_hot.sum(dim=dims)
dice = (2 * intersection + eps) / (denom + eps)eps for absent classes
return 1 - dice.mean()macro Dice
def combined_loss(logits, targets, num_classes, lam=1.0):
ce = F.cross_entropy(logits, targets)(N, C, H, W) and (N, H, W)
dc = dice_loss(logits, targets, num_classes)
return ce + lam * dc, {"ce": ce.item(), "dice": dc.item()}

Fine-tune Mask R-CNN#

model = maskrcnn_resnet50_fpn_v2(weights=MaskRCNN_ResNet50_FPN_V2_Weights.DEFAULT)
in_features = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)counts background
in_features_mask = model.roi_heads.mask_predictor.conv5_mask.in_channels
model.roi_heads.mask_predictor = MaskRCNNPredictor(in_features_mask, 256, num_classes)
for p in model.backbone.parameters():
p.requires_grad = Falsefreezes the FPN too
loss_dict = model(images, targets)train mode: dict of losses
losses = sum(loss for loss in loss_dict.values())
binary_masks = (p['masks'] > 0.5).squeeze(1)eval mode: (N, H, W)

Tasks in code#

TaskCode
Pairwise box IoUbox_iou(boxes_a, boxes_b)
Head output to boxespostprocess(pred_tensor, anchors, stride)
IoU for each classiou_per_class(logits, targets, num_classes)
Crop a box from a feature maproi_align_single(feature, box)
Heatmaps to (x, y)heatmap_to_coords(heatmaps)
Sub-pixel keypointssubpixel_refine(heatmaps)
Split a sentence into conceptssplit_concepts(sentence)
SAM 3 text promptprocessor.set_text_prompt(inputs, "<concept>")

Defaults#

SettingDefaultNote
iou_threshold0.45NMS for object detection
conf_threshold0.25Objectness times best class
lambda_coord5.0Box loss weight, YOLOv1 ratio
lambda_noobj0.5Empty cells dominate otherwise
RPN NMS IoU0.7After the top ~1,000 boxes
Mask threshold0.5Float mask to binary mask
Heatmap sigma2.0Gaussian blob per keypoint
Dice weight lam1.0Loss is ce plus lam times dice

Troubleshooting#

SymptomCauseFix
Weak mAP@0.5:0.95 onlyRough box localisationBetter box loss, CIoU or DIoU
High precision, low recallDetector too conservativeLower the confidence threshold
One object, many boxesAdjacent anchors overlapRun NMS at 0.45
99% pixel acc, empty maskBackground dominatesAdd Dice to cross-entropy
mIoU looks fineOne class at 15% hidesReport IoU per class
Checkerboard artifactsTransposed convBilinear upsample + 3x3 conv
Labels shift by onenum_classes without background4 classes use num_classes=5
Overfits on 500 imagesBackbone and FPN trainFreeze model.backbone

Questions#

Why RoIAlign, not RoIPool?
RoIPool rounds coordinates. RoIAlign lifts mask AP by 3-4 points.
Why heatmaps, not direct (x, y)?
The conv map aligns with the spatial output.
Why anchors?
The head predicts small deltas from prior box shapes.
Why skips in U-Net?
The encoder loses edge detail. Skips give back high-resolution maps.
Why must U-Net input divide by 16?
The encoder halves the resolution four times.
Is SAM 3 faster than Grounded SAM 2?
Usually net-neutral or a slight speedup.