Attention all AlphaPilot Contestants,
We would like to make teams aware of some important updates to Test 2 and Test 3 today.
There have been a few significant updates to the Test 3 simulator, and we suggest all teams pull the latest version from the FlightGoggles GitHub page. A few new features were included to make Test 3 development easier:
- Added functionality after a crash to automatically and manually reset the drone’s location back to the initial pose
- Added a downward-facing laser range finder to aid altitude estimation. The laser range finder points in the negative z direction. Its measurements are defined in the drone body frame, and thus range measurements will have a negative value.
- Contestants can now choose to enable stereo camera sensing.
To alleviate some confusion about the format of Test 3, we updated the README file that accompanies the scorer scripts provided in the Challenge Files (Challenge_Leaderboardtest.zip). The readme contains challenge installation instructions, scorer usage guide, and list of allowed and prohibited information for Test 3. The revised version of the README can be downloaded here:
Challenge 3 is modeled to closely resemble a real-world FPV drone racing scenario. As such, participants know the nominal gate locations from their practice rounds. However, at race time, the actual locations of the gates may be slightly perturbed (unknown to the participants).
Similar to human FPV racers, autonomy algorithms should also be able to navigate even when the gates are slightly perturbed. For the challenge, this robustness is addressed as follows: The nominal gate locations are available for the participants to use in their algorithms. But for the evaluation of their algorithms, small perturbations will be added to these nominal gate locations. The resulting perturbed gate locations are used in the evaluation runs, but are not known a priori to the autonomy algorithms. The bounds on the gate location perturbations are known to the contestants and can be read from the ROS parameter server (see README_v2.md).
The figure below shows a snapshot of the Test 3 Challenge Course. Contestants will start in a known starting location in the upper right corner of the figure. Contestants must traverse gates in order along the course marked in red and must finish by passing through the gate in the lower left corner of the figure (gate 6) within a known time limit. Missing a gate does not lead to disqualification, however if contestants miss a gate in the ordering, they forgo the points for that gate and cannot traverse it (for reward) after passing a gate later in the gate ordering. Gate 6 (the finish line) is unskippable and must be traversed in order to complete the course.
In the figure below, gate IDs belonging to gates along the race path are marked in yellow. Gate IDs in blue are not part of the race course and will not change in position. The ordering of the gates to traverse will not change. Other obstacles in the environment will not change. For Leaderboard score submission, teams are given all 25 gate perturbation layouts, and they will self-report their scores to HeroX. For Final Testing, teams are not given gate perturbations, and their algorithms will be run and scored by HeroX. The course order is shown in Table 1.
Gate 10, 21, 2, 13, 9, 14, 1, 22, 15, 23, 6.
Table 1. The ordered list of gates to traverse to successfully complete the course.
Figure: Shows a screenshot of the Leaderboard/Final Testing Challenge Course in MIT’s FlightGoggles Simulation.
Further, we heard team concerns about the Test 2 training data variability and difficulty identifying each gate's flyable area. We've revised this set of data labels to ensure their accuracy and reduce any ambiguity involved in this portion of the qualifier challenge. Still, some of the labels do not follow the ground-truth guidelines accurately, and that should be considered when developing machine vision algorithms. Handling real-world challenges effectively will be critical for success in the AlphaPilot Competition, and similarly, teams need to deal with some flawed ground-truth in Test 2 and sensor noise, control drift, and modeling errors in Test 3. We updated the link with the new labels, and that can be downloaded here:
There are also a few other updates for Test 2 as requested by teams.
- The beta version of Test 2 starter scripts including updated random_submission.json
- The beta version of Test 2 scorer scripts including fixes to the JSON interpreter
Finally, as a reminder, the Q&A session is soon! If there are any outstanding questions on the qualifiers, the Q&A is a great time to get those answered. Please sign-up here if interested.