xAI, Imagine API’yi 2.0’a Yükseltmeye Hazırlanıyor: Görüntü ve Video Tek…·Microsoft MAI-Cyber-1-Flash’ı Duyurdu·Moonshot AI, Kimi K3 Model Ağırlıklarını ve Teknik Raporunu Açık…
ARAşTıRMA · YAPAY ZEKAarXiv:2611.069104 Kas 2026 · v1
FrontierPhysics: Evaluating Agents for End-to-End Physics Research
Bingran You, Wenjun Ke, Ria Rosenauer, Bo Liu, Wenbo Chen, +67 yazar
YAYIN:4 Kas 2026ALAN:cs.AI
Özet
AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
Özetle: AI agents are playing an increasingly important role in physics research.
Özet
AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
Orijinal Özet (İngilizce)
AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
@article{you2026frontierphysics,
title = {FrontierPhysics: Evaluating Agents for End-to-End Physics Research},
author = {Bingran You and Wenjun Ke and Ria Rosenauer and Bo Liu and Wenbo Chen and Min Chen and Shuqi Xu and Yue Ma and Jiechao Feng and Bo Chen and Xiangkai Sun and Jiarui Liu and Jinen Guo and Zeyu Zhou and Xu Huang and Zhuohan Li and Haoyu Li and Heguang Lin and Lijie Ding and Yadong Zeng and Kangqi Zhang and Xinge Liu and Hanwen Liu and Zhenghong Li and Alexander von Recum and Yuchen Zhu and Felix Schwarzfischer and Yizhou Zhao and Jicheng Wang and Binxu Li and Qian Yu and Chan Li and Cheng Dai and Di Wang and Haoyu Zhang and Jiankai Sun and Kaituo Feng and Mian Qin and Ruihong Yin and Yiqun Luo and Weiwei Wu and Shihong Pan and Tsung Yeh Hsieh and Xiao Huang and Yan Zhang and Sihan Li and Zirui Ren and Zengji Tu and Kyoung Whan Choe and Ke Sun and Vincenzo Galia and Vaisakhi Mishra and Jiashu He and Hongyang Wang and Zhiheng Yang and Shubham Gaur and Yifan Gu and Shuyi Wang and Zihao Zuo and Steven Dillmann and Yiyou Sun and Xinyang Han and Hanchung Lee and Molei Tao and Yue Zhao and Shilong Liu and Yuchen Wu and Xuandong Zhao and Xiangyi Li and Florian Brand and Gerbrand Ceder and Hartmut Haeffner},
journal = {arXiv preprint arXiv:2611.06910},
year = {2026},
url = {https://arxiv.org/abs/2611.06910}
}
Henüz yorum yok. İlk yorumu siz yapın!