Link Search Menu Expand Document

Welcome to RealTime QA

Check the latest results Make a new submission

Check out the GitHub Check out the paper

Latest results

Multiple Choice Track

ModelSubmission Time (GMT)OriginalNOTA
Claude-3.5 Haiku + Google Custom Search2025-03-22 03:00:0070.055.0
Claude-3.5 Sonnet + Google Custom Search2025-03-22 03:00:0070.055.0
Gemini 1.5 Flash + Google Custom Search2025-03-22 03:00:0070.035.0
Claude-3.5 Sonnet2025-03-22 03:00:0065.045.0
GPT-4o + Google Custom Search2025-03-22 03:00:0060.045.0
Claude-3.5 Haiku2025-03-22 03:00:0060.045.0
Gemini 1.5 Flash2025-03-22 03:00:0060.045.0
GPT-3.5 Turbo + Google Custom Search2025-03-22 03:00:0055.040.0
GPT-3.5 Turbo2025-03-22 03:00:0050.040.0
GPT-4o2025-03-22 03:00:0045.040.0
Llama3.1-405B-Instruct + Google Custom Search2025-03-22 03:00:0040.040.0
Llama3.1-405B-Instruct2025-03-22 03:00:0035.030.0

Generation Track

ModelSubmission Time (GMT)EMF1
Gemini 1.5 Flash + Google Custom Search2025-03-22 03:00:0025.030.2
GPT-3.5 Turbo2025-03-22 03:00:0015.024.8
GPT-4o2025-03-22 03:00:0015.023.5
Llama3.1-405B-Instruct + Google Custom Search2025-03-22 03:00:0010.023.2
GPT-4o + Google Custom Search2025-03-22 03:00:005.021.0
GPT-3.5 Turbo + Google Custom Search2025-03-22 03:00:005.017.2
Gemini 1.5 Flash2025-03-22 03:00:005.014.1
Llama3.1-405B-Instruct2025-03-22 03:00:005.012.5
Claude-3.5 Haiku + Google Custom Search2025-03-22 03:00:000.015.8
Claude-3.5 Sonnet + Google Custom Search2025-03-22 03:00:000.09.2
Claude-3.5 Sonnet2025-03-22 03:00:000.09.1
Claude-3.5 Haiku2025-03-22 03:00:000.08.7

(see previous results)

Make a new submission

  1. Download the latest set of RealTime QA (link)

  2. Submit your model predictions. (submission form)

    Submission examples (.jsonl file) are available here