Try to add our email address to your email whitelist: email@example.com and firstname.lastname@example.org. Alternatively, you can leave a message below to request manual account activation.
The bounding boxes in the submitted results overlapping the “others” objects with the threshold larger than the threshold or locating in the ignored regions will be filtered out in the evaluation.
NO. If you would like to report results of your algorithm with different settings (e.g., different parameters and training conditions), please use the training and validation sets for this purpose, and only submit one result to the evaluation server.
Similar to the PASCAL VOC dataset, we labeled some “ignore” regions, which indicates the regions containing the objects that are difficult to be annotated due to low resolution or crowd. Meanwhile, some rarely occurring objects (e.g., machineshop truck, forklift truck, and tanker) are labeled as “others”, which are ignored in evaluation.
Thank you! We are aware of some deficiencies in the provided annotations and are still trying to improve these. We do appreciate all kinds of feedback, so please don’t hesitate to contact us to report any findings.
The evaluation results will be displayed at the bottom of the submission page, and the leaderboard will be updated daily.
I you want to rename your algorithm, please contact us.
The picture below is a schematic illustration of the reference frames (red = x, green = y, blue = z):
The results submitted by each team will be scored based on the completeness and ATE accuracy of the trajectories. All the results will be displayed in the live leaderboard. Each trajectory will be scored based on the standard evaluation points, the accumulation of the scores of all these evaluation points is normalized to 100 points to get the final score of the sequence. Each evaluation point can get 0-10 points according to its accuracy.
Of course, we will provide the calibration data of IMU and cameras.
We will provide some sample datasets along with their ground truth collected with the same sensor kit, but the ground truth for the challenge sequences is not available. However, you can submit your own results in the website evaluation system for evaluation.