Develop clear task instructions, reproducible computational environments, reference solutions, and objective evaluation criteria.
Evaluate AI-generated calculations, programming code, simulations, scientific models, and technical deliverables.
Identify incorrect assumptions, numerical inaccuracies, flawed methodologies, physically impossible results, and reproducibility issues.
Assess the scientific validity and technical quality of model-generated outputs.
Establish standards for determining whether solutions are accurate, complete, and scientifically defensible.
Apply computational methods and domain expertise to improve AI performance on complex physical-science problems.
Required Qualifications
Prior paid experience training, evaluating, benchmarking, or improving AI systems on scientific or technical tasks.
Strong practical proficiency in Python.
Deep expertise in at least one relevant physical-science specialization, such as physics, astronomy, cosmology, materials science, chemistry, computational physics, or computational chemistry.
Hands-on experience applying computational methods to modeling, simulation, numerical analysis, data analysis, or reproducible scientific research.
Ability to evaluate technical solutions rigorously and identify scientific, mathematical, and computational errors.
Strong analytical reasoning and technical problem-solving skills.
Public Proof of Technical Expertise
Applicants should be prepared to provide verifiable evidence of their scientific and computational work. Relevant examples include:
Peer-reviewed publications or preprints involving computational methods.
Public simulation, modeling, or scientific analysis code.
Research profiles such as Google Scholar, ORCID, or arXiv.
Contributions to open-source scientific software.
Reproducible notebooks, computational models, datasets, or simulations.
Public benchmarks, experiments, or technical research projects.
Preferred Qualifications
Experience developing AI-training tasks, reference answers, evaluation rubrics, automated graders, or benchmark problems.
Familiarity with scientific computing and machine-learning libraries such as NumPy, SciPy, SymPy, Astropy, PyTorch, and JAX, or comparable tools.
