Test 2 - scorer_scripts

key insights
  • Scorer scripts for submission evaluation.
  • File updated to v3 on 03/06/2019


  • User avatar
    Neil Johnson March 5, 2019, 2:17 p.m. PST
    I'm having trouble getting the scoring scripts to work. I have installed shapely, and that is importing correctly. I run the following command:

    "python3 score_detections.py -g ../training_GT_labels_v2.json -p realdealrobotics_submission_20190305_FixedNulls_run1.json"
    Where the json file is one that I have generated running our algorithm. When I run it, I get the following cryptic error:
    Traceback (most recent call last):
    File "score_detections.py", line 24, in <module>
    coco_score = mAP_scorer.COCO_mAP(GT_data,pred_data)
    File "/home/neil/devel/realdealrobotics_vision/scorer_scripts_v2/scorer.py", line 52, in COCO_mAP
    sumAP += self.IOU_mAP(GT_data, pred_data, iou_threshold)
    File "/home/neil/devel/realdealrobotics_vision/scorer_scripts_v2/scorer.py", line 74, in IOU_mAP
    File "/home/neil/devel/realdealrobotics_vision/scorer_scripts_v2/scorer.py", line 102, in metrics
    tp_fp_cf[:, 2] = np.array(test_boxes)[:, -1]
    IndexError: index -1 is out of bounds for axis 1 with size 0

    Any ideas? I'm struggling trying to figure out if my json file is formatted correctly (since the leaderboard only updates every 24 hours).
    • User avatar
      Pedro Perez March 6, 2019, 10:50 a.m. PST
      I am having the same problem! Did you find out how to solve it?