The William Lowell Putnam Mathematical Competition, known as the Putnam exam, is a prestigious annual mathematics competition for undergraduate students in the United States and Canada. The exam consists of 12 challenging mathematical problems, with participants given 6 hours to complete the exam. A perfect score on the Putnam exam is rare, even among top students, and would be a significant achievement for an AI system, showcasing its advanced mathematical problem-solving capabilities.
Will clear and strong evidence be provided that an AI is capable of achieving a perfect score on the Putnam exam under real-world conditions before January 1st, 2027?
Resolution Criteria:
This question will resolve to "YES" if, before January 1st, 2027, clear and strong evidence is provided that an AI system is capable of achieving a perfect score on the Putnam exam, adhering to the following conditions:
The AI system must demonstrate the ability to solve all 12 problems correctly within the timeframe given to human competitors in the real-world competition.
The AI system's performance should be evaluated using either an actual Putnam exam or a set of "representative problems." Representative problems are a selection of problems that reflect the difficulty, style, and subject matter typically found in Putnam exam problems. The problems must be unsolved by the AI system prior to the evaluation.
Clear and strong evidence of the AI system's capability must be provided through one or more of the following:
a. A detailed, step-by-step explanation of the AI's solutions to the actual Putnam exam or the representative set of problems.
b. A recorded demonstration of the AI system solving the actual Putnam exam or the representative set of problems within the timeframe given to human competitors.
c. Endorsement of the AI's performance by expert mathematicians or the competition organizers, who have reviewed the AI's solutions and confirm that they meet the standards required for a perfect score.The AI system must not have access to any external resources or assistance during the evaluation, including:
a. External computational equipment or devices.
b. Communication with other AI systems, mathematicians, or individuals.
c. Access to solutions or hints for the actual Putnam exam or the representative set of problems.
The question will be resolved using the question creator's discretion, possibly in consultation with experts in mathematics and AI. Additional details or clarifications may be provided as needed.
Update 2025-02-09 (PST) (AI summary of creator comment): Representative Problems Clarification
Definition: Representative problems are those crafted by mathematicians to closely mimic the difficulty, style, and subject matter of the actual Putnam exam.
Rationale: This approach is suggested to avoid issues stemming from the public posting of actual exam answers.
Evaluation Context: In such cases, the AI's performance on these specially designed problems will serve as the basis for resolution, just as an actual Putnam exam would.
1,000
3.00I have to ask if this is intentional:
The AI system's performance should be evaluated using either an actual Putnam exam or a set of "representative problems." Representative problems are a selection of problems that reflect the difficulty, style, and subject matter typically found in Putnam exam problems
Because if it is I think clarification would be in order as to what counts as representative problems.
@zsig The main issue with the Putnam Exam is that the answers are posted online soon after students complete the test. However, if a group of mathematicians wanted to administer the Putnam exam to AI they could come up with questions that have similar difficulty to the actual Putnam exam and give it to the AI. Representative problems is referring to that kind of situation.