Physics PhD Reasoning Benchmark

About this role

We're developing novel methods to evaluate how well AI systems can reason through advanced physics workflows. This project sits at the cutting edge of AI evaluation—where physics domain knowledge meets frontier model assessment. We're seeking Physics PhD holders (or ABDs) with strong conceptual depth and computational modeling experience to help us design and review high-quality physics problem sets that test AI’s capabilities in solving real-world physics challenges. What You'll Do - Design rigorous, graduate-level physics tasks and rubrics to assess AI reasoning across topics like classical mechanics, quantum systems, thermodynamics, and electrodynamics. - Contribute example solutions and grading rubrics that assess reasoning steps, not just final answers. - Collaborate with our internal team and other domain experts to probe the strengths and failure modes of advanced AI models. - Provide expert feedback on task quality, correctness, ambiguity, and conceptual coverage. What We're Looking For - PhD or ABD in Physics, Applied Physics, or a closely related discipline. - Strong ability to distill complex physics problems and guide others through the reasoning. - Demonstrated experience in computational modeling or numerical analysis (e.g., simulations, data pipelines, Monte Carlo methods, FEA). - Familiarity with at least one of: Python, MATLAB, Mathematica, Julia, C++, COMSOL, ROOT, Geant4, or equivalent scientific tooling. - Experience with publishing, teaching, or curriculum design is a plus. Ideal candidates will be comfortable with both problem design and conceptual review, especially for graduate-level tasks. Compensation & Logistics - Hourly Rate: $65–$100/hour depending on experience and task complexity. - Type: Contract, flexible hours (part-time and full-time options). - Location: Remote (US preferred); Bay Area availability is a bonus