ROI Pooling

Region of Interest Pooling

Fundamental technique in object-detection architectures, popularized by classic models like Fast R-CNN.

What it is and Practical Context

ROI Pooling (Region of Interest Pooling) is a crucial operation in neural networks. Its goal is to extract fixed-size feature maps from regions of interest (ROIs) that have varying sizes.

In region-based object detection (Region-Based CNNs), an image first passes through a convolutional network (backbone) that generates a global feature map. In parallel, algorithms suggest “boxes” (ROIs) where objects might be. Because these ROIs have different proportions and sizes, they cannot be directly fed into Fully Connected Layers, which require fixed-size inputs. ROI Pooling solves this by dividing the projected ROI into a fixed grid (e.g., $7 \times 7$) and extracting the maximum value (Max Pooling) from each subdivision.

🧮 Mathematical Foundation (Dense)

Let a feature map be $F \in \mathbb{R}^{C \times H \times W}$ and a region of interest bounded by continuous coordinates $(x_1, y_1, x_2, y_2)$ projected onto the feature map from the original image (with a scaling factor $S$).

The projection undergoes a first rounding (quantization) to obtain discrete coordinates in the feature map:

$$x'_1 = \lfloor x_1 / S \rfloor, \quad y'_1 = \lfloor y_1 / S \rfloor$$

$$x'_2 = \lceil x_2 / S \rceil, \quad y'_2 = \lceil y_2 / S \rceil$$

Eq. 1: Projection of continuous coordinates onto the discrete grid of the feature map.

Where:

• $x_1, y_1, x_2, y_2$: continuous coordinates of the region of interest in the original image.
• $S$: scale factor (stride) between the original image and the feature map.
• $x'_1, y'_1, x'_2, y'_2$: quantized coordinates projected onto the feature map.
• $\lfloor \cdot \rfloor$ and $\lceil \cdot \rceil$: floor (round down) and ceiling (round up) operations, respectively.

The size of the quantized ROI in the feature map is $h_r = y'_2 - y'_1$ and $w_r = x'_2 - x'_1$.

The goal is to obtain a fixed-dimension output $H_{out} \times W_{out}$ (e.g., $7 \times 7$). To do this, we divide the ROI into a fixed-size grid. Each bin $(i, j)$ (where $0 \le i < H_{out}$ and $0 \le j < W_{out}$) covers a sub-window whose bounding coordinates undergo a second rounding:

$$y_{start}(i) = y'_1 + \lfloor i \cdot \frac{h_r}{H_{out}} \rfloor, \quad y_{end}(i) = y'_1 + \lceil (i+1) \cdot \frac{h_r}{H_{out}} \rceil$$

$$x_{start}(j) = x'_1 + \lfloor j \cdot \frac{w_r}{W_{out}} \rfloor, \quad x_{end}(j) = x'_1 + \lceil (j+1) \cdot \frac{w_r}{W_{out}} \rceil$$

Eq. 2: Calculation of vertical and horizontal boundaries for each ROI sub-window (bin).

Where:

• $H_{out}, W_{out}$: desired fixed output dimensions.
• $h_r, w_r$: height and width of the quantized ROI on the feature map.
• $i, j$: indices of the sub-window in the output grid.
• $x_{start}, x_{end}, y_{start}, y_{end}$: boundary coordinates of each sub-window on the feature map.

The Max Pooling operation is then applied to produce the output value in channel $c$:

$$Y_{c, i, j} = \max_{y_{start}(i) \le y < y_{end}(i)} \ \max_{x_{start}(j) \le x < x_{end}(j)} F_{c, y, x}$$

Eq. 3: Max Pooling operation applied to each ROI bin to generate the fixed output.

Where:

• $Y_{c, i, j}$: output pixel value in channel $c$ at grid position $(i, j)$.
• $F_{c, y, x}$: pixel value in the original feature map in channel $c$ at spatial coordinates $(y, x)$.
• $\max$: mathematical operation that selects the maximum value within the sub-window boundaries.

🎯 When to use

• An inherent part of building classic two-stage models like Fast R-CNN.
• Extracting vector representations of specific regions of an image.

⚠️ Trade-offs and Pitfalls: As seen in the math above, the $\lfloor \cdot \rfloor$ and $\lceil \cdot \rceil$ operations introduce quantization errors (misalignment). The actual bounding box position and the grouped pixel positions lose sub-pixel alignment. For high-precision tasks (e.g., instance segmentation), roi-align replaces these roundings with sampling via bilinear interpolation.

Related: object-detection · fast-r-cnn