Submit

Current

Evaluate new models on the held-out test set

OverviewTaskTimelineRulesParticipateLeaderboardSubmitPapersPublicationsOrganisersFAQ

The Kaggle competitions have closed, but you can still see how a submission would have scored on the public leaderboard.

Track

Submissions are limited per day and track. Re-uploading an identical file returns the earlier result and does not count.

What gets scored

  • BalAcc@10, the competition metric: top-10 balanced accuracy over the 50-word vocabulary (chance 0.2).
  • BalAcc@1: the same, counting only the top prediction.
  • OVMI: out-of-vocabulary mutual information, in bits.

The file is the one you would have uploaded to Kaggle: an index column and one score column per vocabulary word, with a row for every test index of the track.

Submitting from the command line

Handy for large Broad files or scripted evaluation. The response is JSON with the scores.

gzip -k submission.csv
curl --data-binary @submission.csv.gz \
  "https://eval.libribrain.com/api/submissions?track=deep"