Basic Information
Sponsor: Facebook
- Competition: Track multiple moving objects in video captured by an unmanned aerial vehicle (UAV). Detailed explanation please see section Vision Task.
- Submission: Online submission through lpcv.ai August 1st 9:00 AM - August 31st 11:59 PM EST. Each team can submit at most once every 24 hours. Solutions are submitted on the user’s profile page.
- Hardware: Raspberry Pi 3B+.
- Software: Standard system image + PyTorch, Built from master.
- Reimbursement policy: The first 15 teams that submit solutions will receive $200 per team.
- Technical Support: Please join the online discussion forum lpcvc.slack.com. Winners will be announced in September 2021.
- Prizes: Winners will be announced online.
1st place: $3,000.
2nd place: $2,000.
3rd place: $1,000.
_____________________________________________________________________________________________________________________________
Vision Task
- Input: Two files:
- one video (MP4)
- labels of the people and the balls (CSV)
The files are given at the same time (at the beginning). The length of the video is approximately two minutes. This solution will be used as the minimum score for winning teams. A team is disqualified if the team’s solution has correctness below this sample solution.
These are four snapshots of a video taken by a UAV. Several people will pass multiple balls of different colors. The people may move and the UAV may also move. Each person is labelled by a number as one of the inputs, as shown in (a). Occlusion is possible; the red ball cannot be seen in (d).
(a) A number is assigned to each person. | (b) The people move and pass the balls. |
(c) The UAV may also move. | (d) Occlusion is possible. The red ball is held by the person with white T-shirt. |
- Output: Answers to the questions.
- Correctness: sum of the scores of all answers.
There will be several testing videos. For each video:
- x = [output_frames]
- y = [answer_frames]
- correct_x = compare(x, y) # given frames & extra/missing frames will be neglected
- score = len(correct_x) / len(y)
correctness = sum(score) / len(input)
- Energy: measured by a power meter. Energy is the cumulative power of time.
- Score: correctness / energy
- Answer: The solution has to decide when balls change hands. The report should mark the frames when one or multiple balls change hands (held by a different person). Do not report when a ball is flying or continues being held by a person. A solution needs to be correct within 5 frames of time.
Frame Number | Yellow | Orange | Red | Purple | Blue | Green |
5 | 0 | 1 | 5 | 2 | 4 | 0 |
30 | 0 | 3 | 5 | 2 | 4 | 0 |
49 | 0 | 3 | 1 | 2 | 4 | 0 |
60 | 0 | 3 | 1 | 2 | 5 | 0 |
- *Sample Solution: You can find more detailed explanation and a sample solution at the 21LPCVC-UAV_VIdeo_Track-Sample-Solution
____________________________________________________________________________________________________________________________
Frequently Asked Questions (FAQ)
- Q: How many balls will be used?
A: At most six. Each ball has a distinct color. Please notice that the word “GoSports” is not printed on the ball.
- Q: How big is each ball?
A: The diameter is 10 inches.
- Q: How many people may be present?
A: At most ten.
- Q: How will the people be distinguished?
A: There will be different video clips with different levels of difficulty. In some video clips, the people will wear different colors of clothes. Some may wear hats or backpacks. In some other video clips, the people may wear clothes of similar colors.
- Q: How will the numbers be specified?
A: The number will be specified by the coordinate of the lower left of the number.
- Q: Will the numbers be specified only at the beginning?
A: The numbers will be specified at the beginning, in the middle of the clip, and at the end the clip.
- Q: What is the height of the UAV?
A: The UAV may move. The data will be captured in an area restricted to no higher than 400 feet by the Federal Aviation Administration (FAA). The actual height will be much lower than 400 feet. Purdue’s Legal Office and Risk Management have reviewed and approved this project.
- Q: Who are the people in the video? What about their privacy?
A: The people are students at Purdue University. All of them have signed the consent form for appearing in the videos.
- Q: Will the video capture anyone that has not signed the consent form?
A: Everyone in the video has signed the consent form.
- Q: The place looks like a parking lot. Will there be vehicles? Will their license plates be seen?
A: Yes, it is a parking lot. It is possible that the videos see vehicles. All the vehicles in the videos belong to the students that have signed consent forms.
- Q: How many answers may be possible?
A: The video has 30 frames per second. For two minutes, there are 3,600 frames. At most six questions (six colors). There are at most 21,600 answers.
- Q: Will all answers be used for grading?
A: Not necessarily. It is possible that some answers are ignored. At least 500 answers will be used for selecting winners. All teams will be evaluated using the same sets of frames and colors. The organizers will not inform the teams which frames will be used. Thus, it is important to detect all events when balls change hands.
- Q: How is the score calculated? If two people stand very close, how do you determine who holds the ball?
A: The questions will be written so that the answers should be obvious for human eyes.
- Q: How do you measure power?
A: A power meter will be used. The Monsoon AAA10F is tentatively selected for this purpose.
- Q: Can I see the code for the referee system?
A: Yes. The code for the referee system will be available before 2021/04/30.
- Q: How many sample videos will be released?
A: At least five. Each is at least one minute long.
- Q: Will you provide training data?
A: Not for 2021. It is unclear how many minutes of training data will be sufficient.
- Q: Can the data be sent to a cloud?
A: No. The embedded computer test environment is disconnected from the Internet.
- Q: Can I log into the embedded computer and install my software?
A: No. The embedded computer is disconnected from the Internet.
- Q: What happens after running one solution?
A: The embedded computer test environment is reset so that all solutions start from the same initial state.
- Q: Will the computation be performed while a UAV is flying?
A: No. The data has been captured in advance and will be given as a file. This is necessary to ensure fairness: All solutions are evaluated using the same data. Also, this allows the referee system to run 24 hours to accommodate many contestants. Also, Purdue’s safety rules require that UAVs be manually controlled.
- Q: Do I have to use the Inforce 6640™ Single Board Computer?
A: Yes. This is an online competition and it is necessary to have the same hardware so that the solutions can be compared fairly.
- Q: What software framework will be used?
A: PyTorch. This is an online competition and it is necessary to have the same software framework.
- Q: How much time should my program run? Is there any restriction?
A: Suppose the length of a video is n seconds. Your program must finish within 10 * n seconds. If a video clip is 120 seconds, your program must finish within 1200 seconds.
- Q: What happens if my program takes shorter?
A: When your program finishes, your program should issue a command to stop power measurement.
- Q: What happens if my program takes longer than the allowed time?
A: If your program takes too long, your solution is disqualified. The referee system has to evaluate many solutions. Thus, the referee system must not allow any single solution to take too long.
- Q: How will the limit of execution time be measured?
A: The power consumption and execution time will be measured by the referee system. It will not be measured by a human pressing a stopwatch.
- Q: What is the flow of information of the referee system?
A:
- Q: Can a solution reduce the number of pixels before detection?
A: Yes.
- Q: Can a solution skip frames before detection?
A: Yes.
- Q: Will the test data for selecting winners be hidden?
A: Yes. The test data will be different from the sample data and will not be publicly available.
- Q: Will the challenge provide a leaderboard?
A: Yes. Please be aware that the leaderboard uses the data that is already available to all teams. The final winners will be evaluated using different data. Thus, it is possible the final winners may be different from the leaderboard. The purpose of this hidden dataset is to encourage general solutions and discourage overfitting of the public data. The data for selecting winners will not be public.
- Q: Will you provide a sample solution?
A: Yes. An open-source solution will be provided before the end of April.
- Q: Who can participate?
A: Anyone in the world can participate as long as this person is not on the restricted list of Embargoed and Sanctioned Countries by the US government. The restrictions are needed because the organizers reside in the US.
- Q: Is the competition restricted to students?
A: The competition is open to anyone (again, with the restrictions by the relevant laws).
- Q: Are there any rules about conflict of interest?
A: The members in the organizing committee are prohibited from joining any team that enters the competition. The sponsoring organizations are allowed to participate and ranked but are not allowed to receive cash prizes.
- Q: Who are the organizers and sponsors?
A: The information is posted at lpcv.ai.
- Q: What is the registration fee?
A: Zero.
- Q: Is there any requirement for open-source?
A: Yes. The challenge aims to promote innovation and exchange new ideas. Thus, winners must open-source before receiving the cash prizes.
- Q: Will anyone check the source code?
A: Yes. The organizers will check reproducibility and readability.
- Q: Why do you use UAVs?
A: UAVs have many applications, such as inspection of bridges, buildings, and land use.
- Q: What is the long-term goal of this competition?
A: Eventually, sophisticated computer vision should run on UAVs directly. UAVs have limited energy, so low-power computing is important.
- Q: Why do you use an Inforce 6640™ Single Board Computer? Why don’t you run on a UAV’s computer directly?
A: The embedded computers of most commercial UAVs are not user-programmable.
- Q: If the long-term goal is to run vision programs on UAVs, should the data be streaming and computation should take as long as the length of the video? Why do you allow solutions that take shorter or longer?
A: The embedded computers of future UAVs may multi-task processing different types of data (in addition to video). Thus, this competition gives contestants the freedom of managing time. If a solution can process video fast, that solution allows the embedded computers to perform other types of computation. This competition does not wish to impose restrictions on how contestants design their solutions.
- Q: If there are multiple balls and only one ball changes hands, should the answer include the holders of the balls that do not change?
A: Yes. The second input file (text) provides the information about the number of balls. Please visit 21LPCVC-UAV_VIdeo_Track-Sample-Solution for more details.
- Q: Do I lose points if my answer contains frames in which no ball changes hands?
A: No. However, if your program has an excessive amount of unnecessary outputs, your program will likely be slow and you will likely lose.
- Q: Is there a limit of time?
A: Your program should finish within n times the length of the video. If the video is one minute long, your program must finish within n minutes. For now, n is set to 5.
- Q: Where can I sign the agreement form?
A: You should sign the agreement form when submitting your solution to any track. Each account only needs to sign it once. The agreement form is a legal binding document, so please read it carefully
- Q: How can we register as a team?
A: One team only needs one registered account. You can indicate your team members when signing the agreement form.
- Q: How can I register for a specific track?
A: Registered accounts are not limited to attend any track. Accounts will be considered as participating a track when submitting a solution