Mask R-CNN

Mask R-CNN (He et al., 2017) is the natural extension of faster-r-cnn, designed not only to detect objects (draw boxes) but to perform Instance Segmentation — identifying the exact pixels that belong to each object individually.

How It Works

The brilliance of Mask R-CNN was maintaining the fast flow of Faster R-CNN and adding a small parallel network (branch) exclusively in charge of predicting a binary mask (pixel by pixel) for the object bounded by the proposed region.

Backbone and RPN: Identical to faster-r-cnn. Extracts features from the image and generates Region Proposals.
RoI Align: Where roi-pooling was previously used (which suffered from severe quantization errors for pixel-level alignment), Mask R-CNN introduced RoIAlign. RoIAlign uses spatial bilinear interpolation to compute the exact value on the feature map with sub-pixel accuracy, maintaining perfect alignment without destructive rounding.
Parallel Heads: The extracted region feeds two independent paths:
- Main Branch: The classic path for classification and bounding box regression.
- Mask Branch: A small fully convolutional network (FCN) that generates independent binary masks for each of the $K$ possible classes.

🧮 Mathematical Foundations (Tripartite Loss)

Training occurs by simultaneously optimizing three distinct tasks through a combined global loss function:

L = L_{cls} + L_{box} + L_{mask}

Eq. 1: Tripartite loss function of Mask R-CNN.

Where:

$L$ : total network loss.
$L_{cls}$ : ROI classification loss (what the object is).
$L_{box}$ : bounding box regression loss (where the strict limits of the box are).
$L_{mask}$ : average binary cross-entropy (BCE) loss specifically focused on predicting the correct mask for the region.

The Mathematics of the Mask Branch

The mask branch generates an output of dimension $K \times m \times m$ , indicating $K$ binary masks of resolution $m \times m$ (one autonomous mask for each of the $K$ known classes).

Unlike traditional semantic segmentation (where all classes compete for the same pixel in a multinomial global Softmax), Mask R-CNN completely decouples mask classification, applying Sigmoid in isolation.

L_{mask} = - \frac{1}{m^2} \sum_{i,j} \left[ y_{i,j} \log \hat{y}_{i,j}^k + (1 - y_{i,j}) \log (1 - \hat{y}_{i,j}^k) \right]

Eq. 2: Isolated pixel-wise binary cross-entropy (BCE) for the true class.

Where:

$k$ : the true (ground truth) class of the object (identified in the training label).
$y_{i,j}$ : true binary label of the pixel ( $1$ if the pixel is the object, $0$ if it is background).
$\hat{y}_{i,j}^k$ : probability (Sigmoid activator output) that pixel $(i,j)$ belongs to the object in the class $k$ mask.
$m^2$ : total number of pixels in the generated mask grid ( $m \times m$ ).

Only the mask associated with the true class of the object ( $k$ ) actively contributes to the $L_{mask}$ loss, totally ignoring the masks generated for the other $K-1$ classes. This decoupling prevents mathematical interference and was the secret to the model’s spectacular segmentation results.

Internal Structure of the Mask Branch

In the reference configuration for the ResNet-C4 backbone, the RoI aligned by roi-align arrives at the mask branch with resolution 14×14 and passes through 4 convolutional layers (3×3, 256 channels, ReLU) that preserve that resolution. A transposed convolution layer (2×2, stride 2) then doubles the resolution to 28×28, and a final 1×1 convolution produces the $K \times 28 \times 28$ output (i.e., $m = 28$ ). At inference, only the channel corresponding to the class $k$ predicted by the main branch is used; the 28×28 mask is resized by bilinear interpolation to the size of the final box and binarized with a 0.5 threshold.

Results and Generality

The paper demonstrates that the framework goes beyond instance segmentation:

State of the art on COCO: outperformed the winners of the COCO 2015 (MNC) and 2016 (FCIS+++) challenges, reaching 35.7 mask AP with ResNet-101-FPN — without any engineering tricks (“without bells and whistles”).
Box detection as a byproduct: even ignoring the mask branch at inference, multi-task training improves box AP by +0.9 points over the equivalent Faster R-CNN — evidence that auxiliary tasks regularize the model.
Human pose estimation: by modeling each keypoint as a one-hot binary mask, the same architecture surpassed the COCO 2016 keypoint challenge winner, running at 5 fps. A single unified network now predicts boxes, masks, and poses simultaneously.
Generalization to Cityscapes: +30% relative improvement over the previous best result; the paper also honestly diagnoses a domain shift in low-data categories (truck, bus, train), where validation AP systematically diverges from test AP.

References and tools

He, K., Gkioxari, G., Dollár, P., Girshick, R. “Mask R-CNN” (2017). arXiv:1703.06870 — original paper, code available in Detectron (FAIR).