Key Responsibilities
Create realistic computational life-science tasks that challenge advanced AI models.
Develop clear task instructions, reproducible computational environments, reference solutions, and evaluation criteria.
Evaluate AI-generated code, scientific analyses, computational models, experiments, and technical deliverables.
Identify invalid biological assumptions, methodological errors, data leakage, statistical issues, and reproducibility failures.
Assess the scientific accuracy, reliability, and methodological rigor of model-generated outputs.
Apply domain expertise to establish objective standards for evaluating computational and scientific solutions.
Provide detailed technical feedback to support improvements in AI model performance across life-science applications.
Required Qualifications
Prior paid experience training, evaluating, benchmarking, or improving AI systems on scientific or technical tasks.
Strong practical proficiency in Python.
Deep expertise in at least one relevant life-science specialization.
Hands-on experience applying computational methods to modeling, simulation, bioinformatics, biomedical analysis, statistics, or reproducible research.
Strong analytical reasoning and scientific problem-solving skills.
Ability to evaluate complex scientific outputs and identify biological, statistical, methodological, and computational errors.
Relevant specializations include
Biology and computational biology
Bioinformatics
Medicine and healthcare
Biomedical science
Neuroscience
Genomics
Ecology
Evolutionary biology
Public Proof of Technical Expertise
Applicants should be prepared to provide verifiable evidence of their scientific and computational work, such as:
Peer-reviewed publications or preprints involving computational methods.
Public research code or bioinformatics pipelines.
Research profiles such as Google Scholar or ORCID.
PhysioNet or other biomedical competition results.
Contributions to open-source scientific software.