Textbook of AI

CodeForces is a Russian competitive-programming platform that hosts twice-weekly contests for an active community of around 600,000 rated participants. Contestants solve 5–8 problems in 2 hours, with problems graded by hidden test suites under strict time and memory limits. The platform assigns each user an Elo-like rating (range typically 800 to 3500+), with named tiers: Newbie, Pupil, Specialist, Expert, Candidate Master, Master, International Master, Grandmaster, International Grandmaster, Legendary Grandmaster.

Since OpenAI's o1 launch in September 2024, CodeForces ratings have been adopted as a headline benchmark for competitive coding by reasoning-model agents. Models compete in real or held-out contests, often with time-isolation to prevent contamination. The headline number is the percentile within the active human rated population (a 2000 Elo rating sits at roughly the 90th percentile; 2400 Elo at the 99.5th percentile).

The benchmark family also includes:

• LiveCodeBench: a continuously-refreshed CodeForces-style problem set from Naman Jain and colleagues (UC Berkeley, 2024), designed for contamination resistance.
• USACO (USA Computing Olympiad): problem set used in the Anthropic Claude 3 evaluations.
• IOI (International Olympiad in Informatics): used in OpenAI's o3 evaluations and Google DeepMind's AlphaCode 2 paper.
• Codeforces problems retroactively graded: several papers retrospectively run models on past contests to compute counterfactual Elo.

Performance trajectory. AlphaCode (DeepMind, 2022) reached roughly 54th percentile on retrospective contest grading. GPT-4 reached ~5th percentile (rating ~800) on naïve evaluation. OpenAI o1 in September 2024 scored at the 89th percentile (rating 1673) , a level corresponding to "Specialist / Expert" tier. OpenAI o3 in December 2024 scored at the >99.7th percentile (rating around 2727, "Grandmaster" tier), comparable to elite human competitive programmers. Gemini 2.5 Pro Deep Think and Grok 4 reach similar Grandmaster-tier ratings. Claude 4.5 Sonnet with extended thinking sits at around 2200–2300 (International Master).

Known issues. Retrospective grading is contamination-vulnerable, every CodeForces problem and editorial is on the open web. Contest-time evaluation mitigates this but is operationally complex. The Elo-percentile mapping also depends on whether the rated population is restricted to active competitors or includes inactive accounts. Some critics argue competitive programming rewards a narrow style of problem-solving that does not transfer to industrial software engineering, which is why SWE-Bench Verified is the complementary benchmark.

Modern relevance. CodeForces percentile is the headline coding-reasoning number on every reasoning-model launch (o1, o3, R1, Gemini Deep Think, Grok 4, GPT-5). It complements SWE-Bench Verified (real-world bug-fixing) by capturing pure algorithmic problem-solving capability.

Reference: CodeForces is operated by Mike Mirzayanov; benchmark adoption was popularised by OpenAI's Learning to Reason with LLMs (o1 system card, 2024). LiveCodeBench: Jain et al., arXiv 2024.