RoI Align

Region of Interest Align

RoI Align is an advanced pooling operation that solves spatial misalignment in object detection and segmentation, being fundamental for models like Mask R-CNN.

The Problem RoI Align Solves

Before RoI Align, the standard architecture (like in Faster R-CNN) used RoI Pooling. The critical problem with RoI Pooling was quantization (rounding).

When a network proposes a region of interest (e.g., a box with floating-point coordinates like x = 10.4, y = 20.8), RoI Pooling does the following:

1. Rounds these coordinates to align perfectly with the integer pixel grid of the feature map (e.g., x = 10, y = 21).
2. When dividing this region into spatial bins (e.g., 7x7), it rounds again to force alignment with the integer grid.

⚠️ This double rounding generates severe spatial misalignment. For classification this is acceptable, but for instance segmentation, where precision must be at the pixel level to generate accurate masks, this error destroys the object boundaries.

How RoI Align Works

RoI Align completely eliminates quantization.

1. Floating Coordinates: It maintains the continuous values of the region of interest.
2. Exact Division: It divides the region into bins maintaining the exact fractional sizes.
3. Bilinear Interpolation: For each bin, RoI Align samples 4 regular points, arranged in a 2×2 sub-grid inside the bin. Since these rarely fall exactly on an integer “pixel”, each point is computed via bilinear interpolation, based on the 4 nearest neighboring pixels in the feature map.
4. Aggregation: The 4 interpolated values of each bin are aggregated by average pooling (or max pooling), producing the final value of that RoIAlign grid cell.

Measured Impact (Mask R-CNN Ablations)

The original paper (He et al., 2017) quantified the effect of RoIAlign in distinct scenarios:

• ResNet-50-C4 (stride 16): +3 mask AP points over RoIPool — with most of the gain at high IoU (AP75), confirming that alignment benefits precisely fine localization.
• ResNet-50-C5 (stride 32): +7.3 mask AP points (+50% relative). The larger the stride, the more destructive quantization is — and the more RoIAlign shines. With RoIAlign, stride-32 features (30.9 AP) surpass stride-16 features (30.3 AP), resolving a long-standing challenge in the field.
• Keypoint detection: +4.4 keypoint AP points even over FPN (fine strides), since point localization is even more sensitive to misalignment than masks.
• Insensitive to design: results do not depend on the number of sampling points or on max vs. average pooling — as long as there is no quantization.

⚠️ Historical detail: RoIWarp (from MNC, Dai et al., 2016) also used bilinear interpolation but kept RoI quantization — and performed identically to RoIPool. The lesson: what matters is not interpolation, it’s eliminating the rounding.

Interview Summary

“RoI Align is a feature extraction technique that uses bilinear interpolation instead of rounding (quantization) to map proposed regions to a fixed-size feature map. It prevents the spatial misalignment that occurred in the classic RoI Pooling, making it crucial for pixel-accurate tasks like instance segmentation in Mask R-CNN.”

References and tools

• He, K., Gkioxari, G., Dollár, P., Girshick, R. “Mask R-CNN” (2017). arXiv:1703.06870 — section 3 (“RoIAlign”) and Table 2c/2d ablations.

Related: Mask R-CNN · object-detection · faster-r-cnn