VisDrone Part
Try to add our email address to your email whitelist: aiskyeye@qq.com and notification@aiskyeye.com. Alternatively, you can leave a message below to request manual account activation.
The bounding boxes in the submitted results overlapping the “others” objects with the threshold larger than the threshold or locating in the ignored regions will be filtered out in the evaluation.
NO. If you would like to report results of your algorithm with different settings (e.g., different parameters and training conditions), please use the training and validation sets for this purpose, and only submit one result to the evaluation server.
Similar to the PASCAL VOC dataset, we labeled some “ignore” regions, which indicates the regions containing the objects that are difficult to be annotated due to low resolution or crowd. Meanwhile, some rarely occurring objects (e.g., machineshop truck, forklift truck, and tanker) are labeled as “others”, which are ignored in evaluation.
Thank you! We are aware of some deficiencies in the provided annotations and are still trying to improve these. We do appreciate all kinds of feedback, so please don’t hesitate to contact us to report any findings.
The evaluation results will be displayed at the bottom of the submission page, and the leaderboard will be updated daily.
I you want to rename your algorithm, please contact us.
How do we handle empty predictions i.e cases in which our model does not predict any bounding box for that image ? The evaluation code assumes at least one prediction made.
If there exists no output detection/tracking result, please provide an empty file.
When is the result submission deadline?
Hi! The deadline is June 30. We have update the important dates: http://aiskyeye.com/challenge/object-detection/
Sorry, I have two more quesions.
In the SUBMIT page , when I choose the ” OBHECT DETECTION” it shows”Your upload quota has been used up.”
But actually I have just downloaded the test dataset, I have never submitted any file.
When I choose the “Single-Object Tracking”, it comes to my accout page. Its really wired. Could u please help me?
My second question is , could you please tell me the files in “VisDrone2019-SOT-test-dev/attributes/*attr.txt” What do they mean,especially the numbers like”0,0,1,0,0,0,1,0,0,0,1,0″. What do these parameters mean respectively. I didn’t find any document to give a definition. thank you
The upload quota will be updated soon, and each user has 5 upload opportunities. Please note that upload quota is shared between different tasks.
A description of the attributes can be found here: http://aiskyeye.com/evaluate/single-object-tracking/
I have two questions.
First, the result format of the crowd counting task is the format of object detection, not the format of mentioned above. Could you tell me the proper format?
Second, the EVALUATE directory tells me “The evaluation code for crowd counting is available on the VisDrone github”, but in this github I can not find eavluation code of crowd counting. Could you please help me ?
Sorry for the troubles and thank you for your feedback. The download link has been updated（http://aiskyeye.com/evaluate/results-format/）. We will upload the evaluation source code to Github soon.
The evaluation code simply calculated the MAE and MSE, and we uploaded it temporarily to this link: http://uavisionvisdrone.com/static/cnt_eval.txt
Hi, I have three questions:
1. If I understand correctly, each company/group has 3 test submissions per algorithm. So if there would be totally different algorithms used (not just some hyperparameters that change), one company/group would have 3 submissions per algorithm?
2. Is it 3 (like in the FAQ) or 5 (like on the submission page) submissions per algorithm?
3. How is the test-dev evaluation working? Just locally on my machine or is there a seperate evaluation page as well?
Thanks in advance!
Hello, we have updated the old FAQ page.
1. In the ECCV2020 challenge, each company/group has 5 test submissions totally. We recommend that each team only participate in one task. If your team wants to participate in multiple tasks, it is recommended to register a new account to get more submission opportunities, or contact us to modify your account upload limit.
2. The upload quota on the submission page is all your submission quota (shared in all tasks)
3. Evaluation is done on our server. You need to download the test set, get the results locally, and submit the test set results to our server. The evaluation will be performed automatically. After the evaluation is completed, you will receive a notification email and the score will be displayed below the submission form.
Hi, I have a questions:
Are all videos in your dataset at the same frame rate?
If the frame rate is the same, what is the frame rate?
If the frame rate is not the same, what are the kinds?
Thanks in advance!
The original frame rate is 24. But when annotating, we extracted a part of the frames
Hello, I had submit for a while, but the result is still not coming out and could you please tell me how to change the institution of my account.
Thanks
Sorry for that. Due to network issues, our evaluation server went down in the past few hours. Now the problem has been fixed, your evaluation score will be displayed on the page later. If you still have problems, please contact us.
Hi, when I open the submit page of Multi-Object Tracking, the results are as follows:
”
Not Found
Apologies, but the page you requested could not be found. Perhaps searching will help.
”
Could you tell me how to solve it?
This bug has been fixed. Thanks for your feedback.
Thank you for solving my problem, I have another problem，what is the video frame rate of your dataset?
I have a question.
The description document of the corresponding method is asked to submit when I submit the results.But in your download link, I don’t find document templates. There is only one download link for the dataset.
Could you help me？
We fixed a permissions issue and can now download normally: http://aiskyeye.com/download/
Hi. I had submitted my predictions along with a description document before the document template was released. May I know to whom I can send the new description document of our method as per the template? Thanks.
Hi. Please send your new description document to tju.drone.vision@gmail.com。
Hi, “Please note that when you submit the results, you must also submit the corresponding method description document. Document templates can be downloaded on the download page. ” in Submit Page
The download page is not found!
We fixed a permissions issue and you can try to refresh the page.
Baidu yun (code: 5vcm) https://pan.baidu.com/s/1aEXQPoKhDIt7gq_TOhOEcA
Google drive
https://drive.google.com/file/d/1Bzvu6WSV0FxZKWyl8G8pcqmLt-ZLfdZq/view?usp=sharing
execuse me, what should I do if i don’t want to show my result on leaderboard?
Tell us your user name, task, file name, and the upload time. NOTE: The score of this submission will also be hidden from your submission page, but your quota will not be restored.
thank you, I have not submitted, I’m just wondering
How to evaluate test-dev result on evaluation server? In webpage you guys said “Number of time for testing on evaluation server for test-dev dataset is unlimited” so it is allowed to test test-dev set right? But I did not see any info. Please let me know how to do it.
You can evaluate the test-dev result on your local machine. The ground truth of test-dev set is available.
Hi,
I haven’t seen any discrimination for short-term and long-term trackers. Please let me know if we are allowed to use a detector/verifier for our short-term tracker in the VisDrone2020 challenge.
Hi, I have a question, I want detailed information on the challenge-set results, including ap0.5, ap0.75, or something, just like the content of your Matlab evaluation tools.
hi…can I have the python version of your evaluation code for Object Detecion Challenge
Sorry, we only provide matlab implementation.
Hello, would you please give me a manual account activation?
Yes. Your account has been activated.
In croud counting,in each line of the txt file,there are 3 values.What does they mean?
They mean frame and coordinates.
In SOT data set, each of the line of the annotation txt file has information about bounding boxes only(ie;4 values indicating x,y coordinates and width&height of bbox).But which class do they represent?
There is no class information in the annotation file as it does not affect the model and algorithm of SOT.
where could I get the frcnn detection results of testset for MOT task？
Hello. We do not provide detection results. You need to use your own detector.
By mistake, I did submit result with my personal account not with our team account. I ask if there are any disadvantages.
Hello. Your submission can be marked as invalid. You can continue to use the team account to submit results. There will be no disadvantages.
Hello, where is the task4b’s leaderboard of MOT?
Sry, I have a quesiton. Is it normal that I can’t see any submitted result on leaderboard of single-object tracking task? I can’t see myself too.
Pacino Suen is on the leaderboard😂. Leaderboards are not updated in real time.
Hello,what time is the deadline? 0:00 or 24:00 in June 30？and Can I revise my institution because my institution is wrong when i registered my account
We use UTC time. The submission system will close at 24:00 on the deadline. Note that due to the impact of COVID-19, the submission deadline is delayed until July 15th. And each team will have additional 5 submission opportunities.
And you can leave your institution here, we will assist you to revise it.
Thank you! The new insitition is VIONVISION
Hello, can I manually annotate the VisDrone2020 train set in other ways for assistance?
hello, I would know please how much time the upload state takes approximately ? I did upload my submission this morning, and after 3 hours now the statut does not change.
Sorry for that. Due to network issues, our evaluation server went down in the past few hours. Now the problem has been fixed, your evaluation score will be displayed on the page later. If you still have problems, please contact us.
Hi, I’ve submitted two results for hours but the status is still “upload”
Sorry for that. Due to network issues, our evaluation server went down in the past few hours. Now the problem has been fixed, your evaluation score will be displayed on the page later. If you still have problems, please contact us.
Hi Visdrone Team,
I am submitting the results for crowd counting task. I am following the exact format frame number and count in each line of a text file. However, I am getting the format error again and gain. Please help
Hi. Please send your results *.zip file to tju.drone.vision@gmail.com.
Many thanks
Is it possible to use the information of the previous frame and the next frame for crowd counting task？
Sure.
Hello, I am a member participating in 2020SOT. I submitted the zip in the format you gave, but the total submission was unsuccessful. Trouble to ask why. See the attachment for a schematic diagram of the submission format. Thank you.
Hi. Please send your results *.zip file to tju.drone.vision@gmail.com
When I send an email to your mailbox, it shows that the recipient’s address is wrong or does not exist, the mail cannot be delivered, and the first reinvestment is in progress. I look forward to your reply.
I can see having validation and test-dev datasets excluded from the training process for debugging/offline evaluation purposes. Regarding the final submission on the test-challenge set, are we supposed to train on the whole dataset including all the available annotations (train, val and test-dev) or we should rely on the trainining-set split explicitly?
Hello, due to the impact of COVID-19, the submission deadline is delayed until July 15 th. So will the deadline for submitting papers be delayed?
I summited the result. However, I haven’t received the evaluation results, and the status is upload.
Thanks. The evaluation server is running now.
Hello, I have a question about the paper submitition. If I want to submit a paper and have good results on serveral standard datasets, do I have to join your challenge and have results on visdrone dataset, or I just need to fit in one of your topics, for instance, people and object tracking?
Can I change the organization when submit?
Leave a comment here and we will help you change the information
can test-dev set be used as the training data?
any one can answer?
Can test-dev set be used for training ?And how do you check that the participants are not using data sets from other tracks.
I have uploaded my new update (results and description) of the crowd counting algorithm, but the system not giving me the evaluation results. I uploaded much time, one of them in the status ”processing”, the others still “upload”
New organization name: “MCC LAB”, thank you!
Hello, I uploaded my file for object detection sometime back and the interface is showing status as “upload” and score as “N/A”. Could I please know how much time the server takes for evaluation?
The server room where our evaluation server is located is shut down for maintenance at 13:30-17:30, July 13, China Standard Time.
What is the challenge deadline? 23:59, July 14, China Standard Time or 23:59, July 15, China Standard Time?
Our evaluation server will be closed at 23:59 on July 15 (UTC+0 time).
I am trying to upload results for evaluation on crowd counting task. This is my first time to submit here but I am getting message that I used up my quota. Please help!
Nevermind, I was able to submit now. Thank you.
Hello could please please set my institution name to VCL-CERTH? Thank you in advance.
OK.
What does score metric mean for submitted results for task3:MOT3
Higher or lower is better?
Sorry I can’t update my result
the website shows “500 Internal Server Error”
me too~~~
I have the same problem.
Sorry for that. Our server was down in the past few hours and the problem has now been resolved.
can not submit which make me so frustrated
Sorry for that. Our server was down in the past few hours and it has now been resolved.
I can not upload my final submission to the leaderboard due to the server currently being down.
Sorry for that. Our server was down in the past few hours and it has now been resolved.
I want to upload my result, and the website shows ‘500 Internal Server Error’. I remember the deadline is 23:59 on July 15 (UTC+0 time), instead of July 15 (UTC+8 time).
Sorry for that. Our server was down in the past few hours and it has now been resolved.
Sorry to interrupt，how do I get a paper ID?
How do I get a paper ID?
From the CMT system.
how to update my description document, please？
Packed with result files and upload on the submit page.
I summited the result. The website shows “upload”.
I have received the result.
In the leader board window, “Please note that this automatically generated leaderboard is not used as the final ranking of the VisDrone2020 Challenge.” I saw the phrase. Are there other factors affecting final ranking?
I got no verification email after registration.
Dear committee, when will the final ranking be available?
When will the results be released?
I can’t receive the activation email after registration even if I add your both email address to my email whitelist.
Could you please activate my account? The link I received to do so is not working.
I didn’t receive the activation email after registration even after adding both email addresses to my email whitelist. Could someone manually activate my account? Thank you!
Are there public papers on algorithms listed on VisDrone 2020 Leaderboard? thanks.
请问该数据集中无人机拍摄高度大约为多少米？镜头旋转角度大约为多少？有没有甚么方式可以求得视野下面积？（为了求人群密度用）
Hello, is the deadline of VisDrone 2021 already announced?
The deadline is 07 15
when will the leaderboard be released
Sorry, I have kept you waiting, the leaderboard has been updated at http://aiskyeye.com/leaderboard/
There are many problems:
1. Whether the pdf version of the description cannot be included in the submitted documents, otherwise there will be errors in the documents;
2. Why does the provided counting document template include 00011.tex file
Thank you for your feedback. The first problem you mentioned has been solved, but the second problem you mentioned is not understood. Could you please tell me which counting document template refers to?
Hi, the uploaded file exceeds the file size limit, what is the max size of the uploaded file?
The upload interface limit size is unified to 10M
Can we use the test-dev dataset as part of the training set
The test-dev dataset is NOT allowed to be part of the training set.
Hi, when is the deadline for paper submission? is it the same as the deadline for the challenge (15th July)?
One week after the iccv result came out
In the register rules, we have:
“`
1. Please register by the team leader first, choose Role as the team leader, and then let your teammates register, fill in the Team leader Username, and only the team leader has the qualifications for submission.
“`
– It means only leader can submit the result?
– Can member submit other challenge which is different from leader’s challenge?
Hello, first, only the results submitted by the team leader are valid. Second, if you want to participate in other tracks, please contact us to increase your quota.
Where can I modify my team information?
Please send the information that needs to be modified to my mailbox, and I will help you modify it
I have uploaded my new update.but all the status of my submits are upload. what happens?
In this case, the server may be down, which has now been processed.
Hi,
I have created the submission file for the crowd counting task as per the guidelines but the server message says format error. I have tried submitting multiple times still it shows the same error. I have mailed you my result file, please look into it.
Hello, I just checked it for you. The content of the file you submitted is incorrect. Please check carefully. For example, you are missing the answer of 001091.
Could you please tell us the approximate range of the altitude/height from which the images/data was captured using the drone.Even a rough approximation is sufficient.
Thankyou
No record
Will the first few teams be awarded certificates in the competition? That would be nice.
Sorry, I can’t recieve activation email. Please help me.
It has been confirmed for you, and please do not register for multiple submissions.
My email address can’t be activated. Can you help us?
It has been confirmed for you, and please do not register for multiple submissions.
Thanks for your reminder. I use this organization account for the final submission.
Another problem is that if I can’t receive your email,how can I know what to do after the competition.
You can add me WeChat: violin550
Hi, I have encountered some problems with the results I submitted on the MOT. For example, when viewing locally, my tracker visualization is very good, but the evaluation indicators after submitting to the platform are very low. I checked the format that needs to be submitted and found no problems. I really want to ask what the evaluation mechanism of the platform is, the target_id needs to be set from 0, or whether there will be some evaluation errors on the platform. If you can see these, I very much hope to give me some replies and suggestions.
https://github.com/VisDrone/VisDrone2018-MOT-toolkit/issues/1
For the counting task, can we utilize external datasets for pre-training? such as the VisDrone2020-counting dataset.
Why the status keeps being ‘upload’ without processing?
I had submitted object detection session but, status still be upload
please check this
Thank you!
when I submit my result to the evaluation system, I found my submission is in updated status over one hour. I don’t know the reason for this, maybe you can solve my problem.
Hi,
While submitted the results, it displayed “uploaded successfully” but under YOUR SUBMISSIONS section it shows “upload” under the status bar and “N/A” under the score bar. How should i know the exact status of submission and the scores? Should i wait or re-submit?
Score has been updated now. Thanks
Hello,
Is a last submission only valid?
Or the highest score in 5 submissions finally recognized?
Highest score