Link Search Menu Expand Document

Welcome to RealTime QA

Check the latest results Make a new submission

Check out the GitHub Check out the paper


Latest results

Multiple Choice Track

ModelSubmission Time (GMT)OriginalNOTA
GPT-3 + Google Custom Search2022-11-19 03:00:0053.353.3
RAG + Google Custom Search2022-11-19 03:00:0050.040.0
GPT-3 (Closed-Book)2022-11-19 03:00:0050.040.0
GPT-3 + DPR2022-11-19 03:00:0046.743.3
T5 (Closed-Book)2022-11-19 03:00:0046.733.3
RAG + DPR2022-11-19 03:00:0026.730.0

Generation Track

ModelSubmission Time (GMT)EMF1
GPT-3 (Closed-Book)2022-11-19 03:00:0033.346.3
GPT-3 + Google Custom Search2022-11-19 03:00:0033.345.7
GPT-3 + DPR2022-11-19 03:00:0026.743.0
T5 (Closed-Book)2022-11-19 03:00:0016.723.3
RAG + Google Custom Search2022-11-19 03:00:0010.017.2
RAG + DPR2022-11-19 03:00:003.36.7

(see previous results)

Make a new submission

  1. Download the latest set of RealTime QA (link)

  2. Submit your model predictions. (submission form)

    Submission examples (.jsonl file) are available here