Link Search Menu Expand Document

Welcome to RealTime QA

Check the latest results Make a new submission

Check out the GitHub Check out the paper

Latest results

Multiple Choice Track

ModelSubmission Time (GMT)OriginalNOTA
GPT-4o + Google Custom Search2025-05-17 03:00:0075.065.0
Claude-3.5 Haiku + Google Custom Search2025-05-17 03:00:0070.080.0
GPT-4.1 + Google Custom Search2025-05-17 03:00:0070.055.0
Gemini 1.5 Flash + Google Custom Search2025-05-17 03:00:0065.065.0
Claude-3.5 Sonnet + Google Custom Search2025-05-17 03:00:0065.060.0
Claude-3.7 Sonnet + Google Custom Search2025-05-17 03:00:0065.060.0
Llama-4-Scout-17B-16E-Instruct + Google Custom Search2025-05-17 03:00:0060.060.0
Llama-4-Maverick-17B-128E-Instruct + Google Custom Search2025-05-17 03:00:0060.055.0
Claude-3.5 Haiku2025-05-17 03:00:0055.065.0
GPT-4.12025-05-17 03:00:0055.050.0
Gemini 1.5 Flash2025-05-17 03:00:0045.040.0
Claude-3.7 Sonnet2025-05-17 03:00:0045.030.0
GPT-4o2025-05-17 03:00:0040.040.0
Llama-4-Maverick-17B-128E-Instruct2025-05-17 03:00:0040.040.0
Claude-3.5 Sonnet2025-05-17 03:00:0035.035.0
Llama-4-Scout-17B-16E-Instruct2025-05-17 03:00:0030.030.0
Gemini 2.0 Flash + Google Custom Search2025-05-17 03:00:000.00.0
Gemini 2.0 Flash2025-05-17 03:00:000.00.0

Generation Track

ModelSubmission Time (GMT)EMF1
Llama-4-Scout-17B-16E-Instruct + Google Custom Search2025-05-17 03:00:0025.031.7
Gemini 1.5 Flash + Google Custom Search2025-05-17 03:00:0015.022.8
Gemini 2.0 Flash + Google Custom Search2025-05-17 03:00:005.010.0
Llama-4-Scout-17B-16E-Instruct2025-05-17 03:00:005.09.8
GPT-4o2025-05-17 03:00:005.09.8
GPT-4.12025-05-17 03:00:005.08.1
Gemini 2.0 Flash2025-05-17 03:00:005.07.8
Gemini 1.5 Flash2025-05-17 03:00:005.07.0
Claude-3.7 Sonnet + Google Custom Search2025-05-17 03:00:000.012.9
GPT-4o + Google Custom Search2025-05-17 03:00:000.011.0
Claude-3.5 Haiku2025-05-17 03:00:000.010.2
Llama-4-Maverick-17B-128E-Instruct + Google Custom Search2025-05-17 03:00:000.09.9
GPT-4.1 + Google Custom Search2025-05-17 03:00:000.09.1
Claude-3.5 Haiku + Google Custom Search2025-05-17 03:00:000.09.0
Claude-3.5 Sonnet + Google Custom Search2025-05-17 03:00:000.08.6
Claude-3.5 Sonnet2025-05-17 03:00:000.04.5
Llama-4-Maverick-17B-128E-Instruct2025-05-17 03:00:000.01.6
Claude-3.7 Sonnet2025-05-17 03:00:000.00.6

(see previous results)

Make a new submission

  1. Download the latest set of RealTime QA (link)

  2. Submit your model predictions. (submission form)

    Submission examples (.jsonl file) are available here