How you will create impact
The AI Observability Engineer plays a critical role in interaction with our AI product teams to ensure the AI solutions are aligned with the enterprise Responsible AI principles and standards. Additionally, the AI Observability Analyst supports the design and execution of the overall AI Governance program as well as the communication and awareness initiatives associated with Responsible AI principles and methods through the following activities:
Model Performance Monitoring: Tracking ML,LLM, Agentic AI key performance indicators (KPIs) like context relevance, accuracy, precision, recall, and F1 scores to ensure model effectiveness
Data Quality and Bias Detection: Monitoring data inputs for anomalies, bias, and drift, which can impact the fairness and accuracy of AI systems
Utilize observability tools to produce structured, trustworthy insights from detailed data sets on AI system activities.
Manage on-going operations of our AI model management platform and pipelines including evolving our KPIs and model observability capabilities
Review AI solutions across the business to ensure alignment with AIG’s Organizational Policies, AI Governance Framework and Responsible AI principles
Provide support to AI product teams, conduct independent reviews, responding to ad-hoc queries, and participating in workshops
Collaborate with legal, compliance, security, and other teams to coordinate the review of Responsible AI issues, ensuring a cohesive approach across the organization
Create executive-level reports on Responsible AI and provide strategic recommendations to senior leadership
Create and update RAI training content for technical and non-technical audiences at all levels of seniority
Contribute to the development and improvement of RAI assets: methods, technical tutorials, tools
Monitor and remain up to date on emerging global AI regulatory frameworks, standards and policies by participating in workshops, reading publications, networking with the goal of helping to translate into RAI framework updates
What You’ll Bring
Required Skills/Experience
Minimum 3 years’ experience and a Bachelor’s degree is required.
Machine Learning, Large Language Models knowledge of various models, algorithms, and evaluation metrics is crucial for effective monitoring.
Data pipelines and analysis skills with the ability to handle, clean, and transform data for monitoring and analysis.
Excited about the problem of scaling human oversight of AI systems and familiar with LLMs development
Comfort with ambiguity and curiosity as our team is scaling with strong ownership
You bring experience and passion in Responsible AI or AI ethics, with an understanding of AI technologies and frameworks such as Generative AI
You have a strong ability to collaborate with cross-functional teams and drive the implementation of Responsible AI practices, ideally within financial services or other regulated industries
You have a working knowledge of effective policies and processes around the usage of emerging AI tools and products, bringing best practices from the industry
You have a proven track record of solving complex problems, developing mitigation strategies, and influencing others on key AI initiatives
You are passionate about staying ahead of emerging global AI regulations and translating these frameworks into practical, actionable updates
Nice to have
Prior experience building production LLM, RAG or model inference monitoring pipelines
Instrumenting applications using OpenTelemetry distributed tracing or structured logs.
Hands on experience with AI Observability tools such as Fiddler, Arize, LangSmith or similar tools.
Ability to build dashboards and alerts tied to AI model behavior beyond system uptime.
#LI-PS1
At AIG, we value in-person collaboration as a vital part of our culture, which is why we ask our team members to be primarily in the office. This approach helps us work together effectively and create a supportive, connected environment for our team and clients alike.
Enjoy benefits that take care of what matters
At AIG, our people are our greatest asset. We know how important it is to protect and invest in what’s most important to you. That is why we created our Total Rewards Program, a comprehensive benefits package that extends beyond time spent at work to offer benefits focused on your health, wellbeing and financial security—as well as your professional development—to bring peace of mind to you and your family.
Reimagining insurance to make a bigger difference to the world
American International Group, Inc. (AIG) is a global leader in commercial and personal insurance solutions; we are one of the world’s most far-reaching property casualty networks. It is an exciting time to join us — across our operations, we are thinking in new and innovative ways to deliver ever-better solutions to our customers. At AIG, you can go further to support individuals, businesses, and communities, helping them to manage risk, respond to times of uncertainty and discover new potential. We invest in our largest asset, our people, through continuous learning and development, in a culture that celebrates everyone for who they are and what they want to become.
Welcome to a culture of inclusion
We’re committed to creating a culture that truly respects and celebrates each other’s talents, backgrounds, cultures, opinions and goals. We foster a culture of inclusion and belonging through learning, cultural awareness activities and Employee Resource Groups (ERGs). With global chapters, ERGs are a cornerstone for our culture of inclusion. The talent of our people is one of AIG’s greatest assets, and we are honored that our drive for positive change has been recognized by numerous recent awards and accreditations.
AIG provides equal opportunity to all qualified individuals regardless of race, color, religion, age, gender, gender expression, national origin, veteran status, disability or any other legally protected categories.
AIG is committed to working with and providing reasonable accommodations to job applicants and employees with disabilities. If you believe you need a reasonable accommodation, please send an email to candidatecare@aig.com.
Functional Area
DT - Data
AIG PC Global Services, Inc.
