MOROCCO 2018  Field Test —
The DLR Morocco-Acquired Dataset of Mars-Analogue eXploration (MADMAX)

List of all available Data. Locations A-H from the sites:

• Rissani 1 (N 31.2983, W 4.3871) –> A,B,C
• Kess Kess (N 31.3712, W 4.0518) –> D,E
• Maadid (N 31.5005,  W 4.2202) –> F,G,H

Location Details

Dataset Structure

All Data clustered by Location X and Run N.

Each Run X-N has the following data:

.
├── navigation_evaluation.zip
│ ├── orbslam2_nav.csv
│ ├── orbslam2_nav_aligned.csv
│ ├── vins_mono_nav.csv
│ └── vins_mono_nav_aligned.csv
│
├── imu_data.csv
│
├── metadata.txt
│
├── img_rect_left.zip
│ ├── img_rect_left_*.png
│ └── …
├── img_rect_right.zip
│ ├── img_rect_right_*.png
│ └── …
├── img_rect_color.zip
│ ├── img_rect_color_*.png
│ └── …
├── img_rect_depth.zip
│ ├── img_rect_depth_*.png
│ └── …
├── img_omni_up.zip
│ ├── img_omni_up_*.png
│ └── …
└── img_omni_down.zip
├── img_omni_down_*.png
└── …

Calibration

The callibration was done with:
DLR CalDe and DLR CalLab – The open-source DLR Camera Calibration Toolbox

The calibration files contain images of the DLR calibration pattern seen by all cameras. The calibration images are found on the download page, together with the calibration results.

Provided files:

• raw images of the calibration pattern
• calibration results
• calibration information on the rectified images

Go to the download page to access the calibration data:

Benchmark

We use the SLAM navigation algorithms VINS-Mono and ORB-SLAM2 for navigation with MADMAX. You are invited to compare your navigation solution against these two algorithms.

The code: VINS-Mono on Github.com
The paper: VINS-Mono: A Robust and Versatile MonocularVisual-Inertial State Estimator

The code: ORB-SLAM2 on Github.com
The paper: ORB-SLAM2: an Open-Source SLAM System forMonocular, Stereo and RGB-D Cameras

The resulting navigation Data can be downloaded in the download section. The trajectories of VINS/ORBSLAM are shown for each individual experiment under „Location Details“.

The performance of the algorithms is shown here:

Percentage of trajectory completed. Only results with 75%+ completion rate are considered for subsequent analysis.

Navigation Performance: RPE of the navigation algorithms for each experiment.

Navigation Performance: ATE of the navigation algorithms for each experiment.

GNSS Ground Truth:

We computed the ground truth using our own algorithms. However, you can use the provided data of the left, right, and base antenna to compute your own ground truth. The provided raw data can be processed using RTKLIB:
RTKLIB: An Open Source Program Package for GNSS Positioning

Our GNSS setup is shown in the figure below (left). We use two antennas with a baseline of 1.285m to calculate the combined ground truth position and orientation of the Body frame B of SUPER, relative to the Topocentric frame T.

Our algorithmic approach to obtain the ground truth is shown below (right). Everything is described in detail in our paper.

Credits:

This work was funded by the DLR project MOdulares Robotisches EXplorationssystem (MOREX).

This activity has been conducted jointly with the two European Commission Horizon 2020 Projects InFuse and Facilitators. They received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 730068 and 730014.

Publications:

As mentioned above, if you use the data from the data set in your own work, please cite us in the following way:

Meyer, L., Smíšek, M., Fontan Villacampa, A., Oliva Maza, L., Medina, D., Schuster, M. J., Steidle, F., Vayugundla, M., Müller, M. G., Rebele, B., Wedler, A., & Triebel, R. (2021). The MADMAX data set for visual‐inertial rover navigation on Mars. Journal of Field Robotics, 1– 21. https://doi.org/10.1002/rob.22016

List of relevant publications:

• Detailed system description of the Lightweight Rover Unit LRU (the technical reference of SUPER):
  Schuster, Martin J. et al. (2017) Towards Autonomous Planetary Exploration: The Lightweight Rover Unit (LRU), its Success in the SpaceBotCamp Challenge, and Beyond. Journal of Intelligent & Robotic Systems. Springer. doi: 10.1007/s10846-017-0680-9.
  Free Access: https://elib.dlr.de/116749/
• Etna Long Range Navigation Test – Moon-analogue navigation data set with identical sensor setup as SUPER:
  Vayugundla, Mallikarjuna et al. (2018) Datasets of Long Range Navigation Experiments in a Moon Analogue Environment on Mount Etna. In: 50th International Symposium on Robotics, pp. 1-7. ISR 2018; 50th International Symposium on Robotics, 20-21 June 2018, Munich, Germany
  Free Access: https://elib.dlr.de/124514/
• Other publications related to the overall Morocco Field Test campaign:
  • Post, Mark et al. (2018) InFuse Data Fusion Methodology for Space Robotics, Awareness and Machine Learning. In: 69th International Astronautical Congress, Oct 2018, Bremen, Germany; https://hal.laas.fr/hal-02092238
  • Lacroix, Simon et al. (2019). The Erfoud dataset: A comprehensive multi‐camera and Lidar data collection for planetary exploration. Proceedings of the Symposium on Advanced Space Technologies in Robotics and Automation-
    https://www.laas.fr/projects/erfoud-dataset/

Statistics

Length and character of trajectories

Frame Drops

As discussed in the publication, errors with the Ethernet connection of the cameras caused drops of frames.
How much each experiment was affected is shown below.
Interestingly, our evaluation with ORBSLAM2 and VINS-mono did not establish any correlation between frame-drops and navigation accuracy / robustness.

Percentage of frame drops compared to overal number of frames.

Handheld field testing is representative for Rover navigation

As described in our paper (section 6.3), we conclude representativeness of our data set for planetary rover navigation, as the navigation algorithms have the same accuracy compared to similar Rover-based navigation data sets.

Access

To gain access to the full list of sets please fill in.