US: $207000 - $300000 (USD) + 20% bonus target + equity + benefits
Learn more about benefits at Google.
Evaluate and model launch analysis.
Work cross-functionally on data collection, management, quality measurement and improvements.
Improve end-to-end evaluations by incorporating user and product feedback, and providing continuous production evaluation/verification for model releases.
Provide user feedback analysis and insights to improve our frontier models and the user experiences.
Solve complex, high-impact research and engineering challenges where the solution is non-obvious to executive leaders.
Minimum qualifications
Bachelor's degree in Computer Science, Electrical Engineering, Mathematics, Statistics, a related technical field, or equivalent practical experience.
Experience with machine learning.
Experience in software engineering.
Experience in Python programming.
Experience in data analysis and related tools and processes.
Preferred qualifications
Master’s degree or PhD in Engineering, Computer Science, a technical related field, or equivalent practical experience.
Extensive experiences on evaluations and user feedback analysis.
Experience using internal/standard data analysis platforms and interactive notebooks (e.g., Colab).
Strong quantitative and engineering skills and proven knowledge and experience working on large scale data analysis, autorater development, and human-in-the-loop quality improvements.
Savvy users of LLM products (e.g., Gemini, etc.) who is familiar with standard LLM evaluation and testing methodologies.
Bachelor's degree in Computer Science, Electrical Engineering, Mathematics, Statistics, a related technical field, or equivalent practical experience.
Experience with machine learning.
Experience in software engineering.
Experience in Python programming.
Experience in data analysis and related tools and processes.