Individual pay is determined by factors including job-related skills, experience, and relevant education or training.
US: $159000 - $230000 (USD) + 15% bonus target + equity + benefits
Learn more about benefits at Google.
Help to establish metrics, benchmarking Generative-AI issues and quantify improvements. Develop datasets and run evals for engineers to evaluate and improve Search, Labs and Assistant products. This role works with sensitive content and may be exposed to graphic, controversial, or upsetting content.
Use technical judgment to drive project delivery, challenge proposals and identify risks. Manage interdependent projects of moderate to high complexity against crashed timelines.
Improve tools through data analysis, technical expertise, and presentations to key stakeholders. Enhance operational workflows via process improvements and identification of automation opportunities.
Design, develop, and scale production-grade LLM-as-a-judge autoraters and automated evaluation systems to run real-time launch decisions and continuous regression testing across core verticals.
Partner directly with Product, Search Core, and GDM engineering teams to drive alignment on technical roadmaps, integrations and cross-product risk registers.
Minimum qualifications
Bachelor's degree or equivalent practical experience.
5 years of experience in data analysis, including identifying trends, generating summary statistics, and drawing insights from quantitative and qualitative data.
5 years of experience in the anti-abuse domain, with a focus on signal analysis, model development, data analysis, and cyber security.
Experience programming in one or more languages (e.g., SQL, Python, or Golang).
Preferred qualifications
Experience performing fraud/abuse investigations, risk management, security, and threat analysis for communication products.
Technical and product experience with end-to-end encrypted communication products and anti-abuse messaging solutions.
Proficiency in large-scale data analysis, tools, and processes for building ML reputation models.
Strong problem-solving, critical thinking, and communication skills with the ability to influence cross-functional stakeholders and senior leadership.
Development of policy and enforcement models focused on actor and behavior-level strategies to identify and prevent harmful patterns, and privacy and safety-focused product design protections.
Bachelor's degree or equivalent practical experience.
5 years of experience in data analysis, including identifying trends, generating summary statistics, and drawing insights from quantitative and qualitative data.
5 years of experience in the anti-abuse domain, with a focus on signal analysis, model development, data analysis, and cyber security.
Experience programming in one or more languages (e.g., SQL, Python, or Golang).