R-CNN

Regions with CNN features

R-CNN was the first major architecture to apply Convolutional Neural Networks to the task of object detection, paving the way for models like fast-r-cnn.

How it Works

The R-CNN architecture (Girshick et al., 2014) operates in four sequential steps:

1. Region Proposal: Uses an external, non-neural algorithm called Selective Search to find about 2,000 regions (bounding boxes) that possibly contain objects in the image.
2. Warping: Since CNNs require fixed-size inputs, each of these 2,000 regions is warped to a standard size (e.g., 227x227).
3. Feature Extraction: The major breakthrough: each of the 2,000 regions is passed independently through a CNN (usually AlexNet) to extract a feature vector.
4. Classification and Regression:

   • The feature vector is passed to linear SVMs (one for each class) to classify the object.

   • The vector also feeds a linear regressor that fine-tunes the bounding box coordinates. This regressor learns to transform the initial proposal $P$ into the predicted ground truth box $\hat{G}$ through translations and log-space scale transformations:

     $$\hat{g}_x = p_w d_x(P) + p_x, \quad \hat{g}_y = p_h d_y(P) + p_y$$

     $$\hat{g}_w = p_w \exp(d_w(P)), \quad \hat{g}_h = p_h \exp(d_h(P))$$

     Eq. 1: Translation and scale transformations in R-CNN's linear Bounding Box regression.

     Where:

     • $\hat{g}_x, \hat{g}_y, \hat{g}_w, \hat{g}_h$: coordinates of the center $(x,y)$, width, and height of the refined bounding box.
     • $p_x, p_y, p_w, p_h$: coordinates of the center $(x,y)$, width, and height of the original region proposal $P$.
     • $d_x, d_y, d_w, d_h$: linear transformations learned by the regressor (the predicted offsets or deltas).

The Problem with R-CNN (Why it evolved?)

Despite being revolutionary in accuracy, R-CNN had critical performance problems:

• ⚠️ Extremely Slow: Because the CNN runs independently for ~2,000 regions per image, there is no computation sharing. Inference took almost 50 seconds per image.
• ⚠️ Complex Training: It was a multi-stage pipeline (train the CNN, then SVMs, then regressors). It was not an end-to-end trained model.

Related: fast-r-cnn · object-detection