Quotes are from the above mentioned Zisserman paper – 4.2 Evaluation of Results (Page 11):
First an “overlap criterion” is defined as an intersection-over-union greater than 0.5. (e.g. if a predicted box satisfies this criterion with respect to a ground-truth box, it is considered a detection). Then a matching is made between the GT boxes and the predicted boxes using this “greedy” approach:
Detections output by a method were assigned to ground truth objects
satisfying the overlap criterion in order ranked by the (decreasing)
confidence output. Multiple detections of the same object in an image
were considered false detections e.g. 5 detections of a single object
counted as 1 correct detection and 4 false detections
Hence each predicted box is either True-Positive or False-Positive.
Each ground-truth box is True-Positive.
There are no True-Negatives.
Then the average precision is computed by averaging the precision values on the precision-recall curve where the recall is in the range [0, 0.1, …, 1] (e.g. average of 11 precision values). To be more precise, we consider a slightly corrected PR curve, where for each curve point (p, r), if there is a different curve point (p’, r’) such that p’ > p and r’ >= r, we replace p with maximum p’ of those points.
What is still unclear to me is what is done with those GT boxes that are never detected (even if the confidence is 0). This means that there are certain recall values that the precision-recall curve will never reach, and this makes the average precision computation above undefined.
Edit:
Short answer: in the region where the recall is unreachable, the precision drops to 0.
One way to explain this is to assume that when the threshold for the confidence approaches 0, an infinite number of predicted bounding boxes light up all over the image. The precision then immediately goes to 0 (since there is only a finite number of GT boxes) and the recall keeps growing on this flat curve until we reach 100%.