Traffic signs are important road features that provide navigation rules and warnings to drivers. Their detection and identification in image is largely investigated by the ADAS (Advanced Driver Assistance System) research community since the 90s. Many different pattern recognition techniques are applied for this purpose.

Most of the traffic sign extraction systems deal with detection and classification of signs in single images. Some systems integrate the detection and classification in a tracking mode. Consequently, more reliable decisions can be made using multi-frame rather than single-frame information. Overall, there is a large amount of research work on detection and classification of traffic signs using single or multi-frame image data. In contrast, fewer authors investigated 3D reconstruction and localization of traffic signs, although, it is required for several applications:

1. Road inventory and update,
2. Visibility analysis in urban areas,
3. Virtual reality and 3D city modeling,
4. Vision based positioning using visual landmarks.

The database contains 1000 images synthesized in a virtual urban environment composed of a 3D city model (Bati3D) enriched with virtual traffic signs positioned at locations known only by the benchmark organizers, and allowing for a low cost perfect ground truth.

The poses used for the synthesis of the images are real poses from a real acquisition by Stereopolis II, a MLS system developed at the French National Mapping Agency (IGN).

• The orientations are given in self explicit xml format.
• All coordinates are geo-referenced (E,N,U) in Lambert 93 and altitude IGN1969 (grid RAF09) reference system.
• The participants are asked to provide the 2D detcted signs and reconstructed 3D signs resulting from reconstruction in xml format (to define)

The benchmark does not aim at ranking the participants but at providing insights on the strengths and weaknesses of each methods. We consider that the quality of a method is subjective and application dependent, and the results of this benchmark should only help a user choosing one approach depending on its own specific requirements. Results will be evaluated at two levels: as a 2D detection, a 3D detection and as a reconstruction. Details of the evaluation metrics used are given in the Evaluation Protocol document.

The benchmark is not yet open and functional. To take part:

Download the Oriented images

Download the Reference signs

Run your algorithms

Send a mail to Bruno Vallet for details on uploading your results

In order to keep informed about the potential modifications or to give some feedback about this benchmark, please contact Bruno Vallet