Object Detection

Type of Computer Vision Tasks

Classification

Object Detection & Instance Detection

Object Detection

Input: Single RGB Image
Output: A set of detected objects;

• For each object predict:
  • Category label (From fixed, known set of categories)
  • Bounding Box (Four numbers: x, y ,width, height)

Challenges

Multiple outputs: Need to output variable numbers of objects per image
Multiple types of output: Need to predict category label and bounding box
Large images: Classification works at 224x224

• Need higher resolution for detection

Detecting a single object

Classification + Localization

Two branches

• Multiple loss functions
• Combine since we only want one loss function for Gradient Descent
  

Detecting multiple images

Need different numbers of outputs per image

Sliding Window

Apply a CNN to many different crops of the image

• CNN classifies each crop as object or background

Region Proposals: Selective Search

• Find small set of boxes that are likely to cover all the objects
  • “blobby” image regions
• Relatively fast to run
  • Gives 2000 region proposal in a few seconds on CPU

R-CNN: Region-Based CNN

Steps

1. Run region proposals
   • Get regions of interest
2. For each region proposals
   1. Warp region to fix size
   2. Run independently through CNN
   3. Classify each region
   4. Bounding box regression
      

Test time

Input: Single RGB image

1. Run region proposal method to compute ~2000 region proposals
2. Resize each region to 224x224 and run independently through CNN to predict class scores and boundary box transform
3. Use scores to select a subset of region proposals to output
4. Compare with ground-truth boxes

Comparing Boxes: Intersection over Union

How can we compare out prediction to the ground-truth box?

IoU > 0.5 is “decent”
IoU > 0.7 is “pretty good”
IoU > 0.9 is “almost perfect”

Overlapping Boxes: Non-Max Suppression (NMS)

Problem: Object detectors often output many overlapping detections

Solution: Post-process raw detections using NMS

1. Select next highest-scoring box
2. Eliminate lower-scoring boxes with IoU > threshold (e.g. 0.7)
3. If any boxes remain, go to step 1

Fast R-CNN

Problem with Slow R-CNN

Very Slow! Need to do ~2k forwards passes for each image!

Faster because it can share computation between image proposal regions

• Swapped CNN with cropping + warping

What does crop features mean?

Region of Interest Pooling

1. Project proposal onto features
2. “Snap” to grid cells
3. Max pool within each subregion

Problem

Misalignment when we snap to grid cells

RoI Align

No “snapping”

Speed Improvement

Faster R-CNN

Insert Region Proposal Network to predict proposals from features
Otherwise same as Fast R-CNN

• Crop features for each proposal
• Classify each one
  

Region Proposal Network

1. Run backbone CNN to get features aligned to input image
2. Image an anchor box of fixed size at each point in the feature map
3. At each point, predict whether the corresponding anchor contains an object
   1. For positive boxes, also predict a box transform to regress from anchor box to object box
      

Jointly train with 4 losses

1. RPN Classification: Anchor box is object / not object
2. RPN Regression: predict transform from anchor box to proposal box
3. Object Classification: Classify proposals as background / object
4. Object Regression: Predict transform from proposal box to object box
   

SPEEDDDDD

Two-stage Object Detector

First stage: Run once per image

• Backbone network
• Region proposal network
  Second stage: Run once per region
• Crop features: RoI pool / align
• Predict object class
• Prediction bbox offset
  

Do we need the second stage?

Single-Stage Object Detection

Two stage more accurate but slow
Single stage faster but not as accurate