Evaluate large language model (LLM)-powered applications using curated test sets, metrics, automated and LLM-based scoring, human review, and regression testing, and use the results to drive improvements
Partner with engineering teams to productionize solutions, and monitor quality, latency, and cost once deployed
Work with product and business partners to translate requirements into AI/ML specifications, define milestones, and deliver using Agile methodology
Apply responsible AI practices, including explainability, and meet model risk and governance requirements in a regulated environment
Required qualifications, capabilities, and skills
Three years of experience developing AI/ML applications, with an advanced degree in Computer Science, Data Science, Statistics, or an equivalent field, or equivalent practical experience
Hands-on experience building applications with large language models (LLMs), including prompt engineering, context management, and output parsing
Experience evaluating AI/ML systems, including designing metrics, test sets, and experiments for non-deterministic outputs
Knowledge of data structures, algorithms, machine learning, data mining, and information retrieval
Hands-on experience applying classical machine learning (for example, linear models, tree-based ensembles, clustering) and deep learning (for example, neural networks, transformers), including feature engineering, model training, and validation
Proficiency in Python with hands-on experience in machine learning and deep learning frameworks (PyTorch, scikit-learn) and generative AI ecosystems (Hugging Face, large language model application programming interfaces, LangChain/Google ADK)
Experience using version control, code review, and testing practices
Collaboration skills and the ability to communicate effectively with both technical and non-technical audiences
Preferred qualifications, capabilities, and skills
Experience with agentic patterns, including tool calling, multi-step orchestration, and human-in-the-loop review
Familiarity with at least one of the following areas: Knowledge Graphs, Computer Vision, Reinforcement Learning, Recommender Systems, or Time Series Analysis
Experience with Amazon Web Services (AWS) Cloud services
Experience with large-scale data processing (for example, Spark)
This position is subject to Section 19 of the Federal Deposit Insurance Act. As such, an employment offer for this position is contingent on JPMorganChase’s review of criminal conviction history, including pretrial diversions or program entries.