Welcome to RealTime QA
Check the latest results Make a new submission
Check out the GitHub Check out the paper
Latest results
Multiple Choice Track
| Model | Submission Time (GMT) | Original | NOTA |
|---|---|---|---|
| llama-4-scout | 2026-06-27 03:00:00 | 25.0 | 10.0 |
| llama-4-maverick | 2026-06-27 03:00:00 | 15.0 | 15.0 |
| llama-4-scout + Google Custom Search | 2026-06-27 03:00:00 | 15.0 | 10.0 |
| llama-4-maverick + Google Custom Search | 2026-06-27 03:00:00 | 15.0 | 10.0 |
| gemini-2.5-pro | 2026-06-27 03:00:00 | 10.0 | 10.0 |
| claude-sonnet-4.6 | 2026-06-27 03:00:00 | 10.0 | 10.0 |
| claude-opus-4.6 | 2026-06-27 03:00:00 | 10.0 | 10.0 |
| claude-sonnet-4.6 + Google Custom Search | 2026-06-27 03:00:00 | 10.0 | 5.0 |
| gpt-5.4 | 2026-06-27 03:00:00 | 5.0 | 10.0 |
| claude-opus-4.6 + Google Custom Search | 2026-06-27 03:00:00 | 5.0 | 10.0 |
| gpt-5.3-chat + Google Custom Search | 2026-06-27 03:00:00 | 5.0 | 5.0 |
| gpt-5.4 + Google Custom Search | 2026-06-27 03:00:00 | 5.0 | 5.0 |
| gemini-2.5-pro + Google Custom Search | 2026-06-27 03:00:00 | 5.0 | 5.0 |
| gemini-3.1-pro-preview | 2026-06-27 03:00:00 | 5.0 | 5.0 |
| gemini-3.1-pro-preview + Google Custom Search | 2026-06-27 03:00:00 | 5.0 | 5.0 |
| gpt-5.3-chat | 2026-06-27 03:00:00 | 0.0 | 10.0 |
Generation Track
| Model | Submission Time (GMT) | EM | F1 |
|---|---|---|---|
| gpt-5.4 + Google Custom Search | 2026-06-27 03:00:00 | 55.0 | 71.0 |
| gpt-5.3-chat + Google Custom Search | 2026-06-27 03:00:00 | 50.0 | 65.3 |
| llama-4-maverick + Google Custom Search | 2026-06-27 03:00:00 | 50.0 | 63.3 |
| gpt-5.3-chat | 2026-06-27 03:00:00 | 45.0 | 61.5 |
| claude-sonnet-4.6 + Google Custom Search | 2026-06-27 03:00:00 | 45.0 | 56.9 |
| llama-4-scout + Google Custom Search | 2026-06-27 03:00:00 | 45.0 | 54.3 |
| llama-4-scout | 2026-06-27 03:00:00 | 40.0 | 55.7 |
| claude-sonnet-4.6 | 2026-06-27 03:00:00 | 40.0 | 53.3 |
| llama-4-maverick | 2026-06-27 03:00:00 | 35.0 | 51.1 |
| gemini-3.1-pro-preview + Google Custom Search | 2026-06-27 03:00:00 | 35.0 | 48.3 |
| gemini-3.1-pro-preview | 2026-06-27 03:00:00 | 35.0 | 46.7 |
| claude-opus-4.6 | 2026-06-27 03:00:00 | 35.0 | 46.1 |
| claude-opus-4.6 + Google Custom Search | 2026-06-27 03:00:00 | 35.0 | 44.6 |
| gpt-5.4 | 2026-06-27 03:00:00 | 30.0 | 46.0 |
| gemini-2.5-pro + Google Custom Search | 2026-06-27 03:00:00 | 25.0 | 30.8 |
| gemini-2.5-pro | 2026-06-27 03:00:00 | 20.0 | 32.5 |
Make a new submission
Download the latest set of RealTime QA (link)
Submit your model predictions. (submission form)
Submission examples (
.jsonl file) are available here