Computer Vision Concept

Computer Vision is the interdisciplinary field of Artificial Intelligence focused on teaching computers to process, interpret, and extract meaning from real-world visual information (such as images and videos). The ultimate goal is to emulate and automate the complex tasks that the human visual system performs effortlessly.

The Semantic Gap

For a human, a photo is instantly recognized as a dog or a landscape. For a computer, that same photo is just a tangle of numerical intensity values. The central challenge of all modern Computer Vision is to overcome this semantic gap — the massive gap between the raw mathematical numbers read by the hardware and the logical meaning abstracted by us.

🧮 Mathematical Foundations (Representation)

In the algorithmic universe, a digital image sheds its visual status and becomes purely mathematical. It is formally treated as a discrete multidimensional tensor.

$$I \in \mathbb{R}^{H \times W \times C}$$

Eq. 1: Tensor representation of a digital image.

Where:

• $I$: the tensor or data structure representing the entire image.
• $H$: height of the image grid in pixels.
• $W$: width of the image grid in pixels.
• $C$: number of color spectrum channels. In monochromatic (grayscale) images, $C=1$. In conventional color images (like RGB), $C=3$.

To access information at a specific point in space, we use the evaluated function notation $I(x, y)$, which returns the color intensity vector (of dimension $C$) existing at the discrete spatial coordinate $(x, y)$. Every convolutional operation (like that of the cnn) consists of mathematically manipulating these tensors.

The Task Hierarchy

The Computer Vision (CV) field solves the problem by dividing it into standardized tasks, progressively more granular and challenging:

1. Image Classification: The analytical foundation. Answers the question “What is this?”. The model receives $I$ and returns a single label (e.g., “it’s a cat”).
2. Object Detection: In addition to saying “what”, it says “where”. The algorithm maps imaginary rectangles (bounding boxes) encompassing multiple objects simultaneously in the same photo. (E.g., faster-r-cnn, YOLO).
3. Semantic Segmentation: Replaces generic boxes with contoured cutouts, classifying each individual pixel of the image into a world category (sky, asphalt, person). However, it does not distinguish individual entities (two people together are seen as a large “person block”). (E.g., u-net).
4. Instance Segmentation: The practical pinnacle of visual understanding. The network cuts out the pixels perfectly, and also differentiates individuals (painting “person 1” and “person 2” as separate and independent entities). (E.g., mask-r-cnn).

Related: cnn