https://www.safe.ai/blog/humanitys-last-exam

Humanity's Last Exam is a new benchmark developed by CAIS and Scale AI to measure AI's proximity to expert-level capabilities. The goal is to create the world's most difficult AI test by gathering complex questions from experts across fields. The project invites experts to submit challenging questions, with accepted contributors offered co-authorship on the resulting paper and prizes from a $500,000 pool.

Currently, to qualify for the benchmark, all models must fail the submitted question.