Key Responsibilities
Conduct independent validation of AI usage in workflows, including testing for accuracy, efficiency of guardrails, orchestration, memory management and token usage.
Design and implement comprehensive validation strategies and test plans tailored to each AI use case. Define clear success criteria and quality benchmarks to measure whether the AI solution meets all requirements and standards.
Understand tool architecture, potential vulnerabilities and weaknesses.
Assess conceptual soundness, methodology, and alignment with intended business use.
Review data quality, preprocessing steps and engineering decisions for AI use cases.
Execute functional and performance tests.
Evaluate performance metrics, robustness, and limitations under various scenarios.
Document validation findings, provide recommendations
Stay current on emerging AI technologies, validation techniques, and regulatory developments.
Communicate validation conclusions to relevant stakeholders, including escalating identified findings, risks or gaps to senior leadership and approval committees
Collaborate with AI Engineering, Data Science, and Risk teams to enhance AI transparency and trustworthiness.
Experience, Skills and Qualification
With 3-7 years’ works experience in the IT or financial industry.
Degree or Advanced degree (MS, PhD) in Computer Science, Data Science, Statistics, Applied Mathematics, or related field.
Strong understanding of generative AI technologies, including large language models (LLMs), retrieval-augmented generation (RAG), agentic systems, prompt engineering, tool use, and modern AI application architectures.
Experience with Python and common AI development libraries and frameworks (PyTorch, TensorFlow or similar).
Excellent analytical, communication, and documentation skills.
Ability to work independently and challenge stakeholders constructively.
Experience with one or more of the following
Validating usage of large language models (LLMs) in business context
Evaluation methodologies, benchmark development, ground-truth creation, and AI quality assessment
AI observability, monitoring, performance measurement, and operational controls
AI orchestration frameworks, agent architectures, prompt engineering, memory management, and context optimization
Exposure to cloud-based AI platforms and scalable model deployment
MCP, APIs, connectors and integrations of AI solutions with enterprise systems and data sources
Responsible AI practices, including fairness, explainability, bias assessment, transparency, and model governance
Fine-tuning or model customization techniques
Security testing and risk assessment of AI systems, including prompt injection, jailbreak, and data leakage testing
Our benefits
To help you stay energized, engaged and inspired, we offer a wide range of benefits including a strong retirement plan, tuition reimbursement, comprehensive healthcare, support for working parents and Flexible Time Off (FTO) so you can relax, recharge and be there for the people you care about.
Our hybrid work model
BlackRock’s hybrid work model is designed to enable a culture of collaboration and apprenticeship that enriches the experience of our employees, while supporting flexibility for all. Employees are currently required to work at least 4 days in the office per week, with the flexibility to work from home 1 day a week. Some business groups may require more time in the office due to their roles and responsibilities. We remain focused on increasing the impactful moments that arise when we work together in person – aligned with our commitment to performance and innovation. As a new joiner, you can count on this hybrid model to accelerate your learning and onboarding experience here at BlackRock.
Guidance on AI use for candidates
At BlackRock, AI has long been part of how we work – enhancing decision-making, improving operations, and helping us deliver better outcomes for clients. We encourage candidates to use AI thoughtfully to learn, prepare, and work more effectively; but during our interview process, we want to focus on getting to know you through your own experiences, thinking, and judgment. To support you, we’ve provided guidance on when and how to use AI during our hiring process so you can approach each step with confidence and showcase your best self.
About BlackRock
At BlackRock, we are all connected by one mission: to help more and more people experience financial well-being. Our clients, and the people they serve, are saving for retirement, paying for their children’s educations, buying homes and starting businesses. Their investments also help to strengthen the global economy: support businesses small and large; finance infrastructure projects that connect and power cities; and facilitate innovations that drive progress.
This mission would not be possible without our smartest investment – the one we make in our employees. It’s why we’re dedicated to creating an environment where our colleagues feel welcomed, valued and supported with networks, benefits and development opportunities to help them thrive.
To learn more about BlackRock, please visit Careers.BlackRock.com. We also encourage you to get to know us on LinkedIn, Instagram, YouTube, X, and TikTok.
BlackRock is proud to be an Equal Opportunity Employer. We evaluate qualified applicants without regard to age, disability, family status, gender identity, race, religion, sex, sexual orientation and other protected attributes at law.
