Responsibilities
Work with users to identify suitable AI applications and define task-level measures of quality, reliability, and operational usefulness.
Build and integrate model APIs, retrieval, structured outputs, and tool-calling agents into application services.
Develop evaluation datasets and regression checks that reflect customer tasks, failure consequences, and changing inputs.
Implement bounded tool access, human review where needed, failure handling, and controls for sensitive data and prompt injection.
Measure and improve retrieval quality, model behavior, latency, and inference cost. Choose models and techniques based on evidence.
Support initial deployments with tracing, monitoring, and investigation of failures and regressions.
Create reusable evaluation and integration components, document limitations, and transfer operating knowledge to the sustaining team.
Qualifications
Required Qualifications
Strong Python and backend engineering skills, including tested services, APIs, version control, and production debugging.
Experience deploying a ML capability into a production application and taking ownership for its behavior after release.
Hands-on experience with LLM applications, including retrieval, structured outputs, tool calling, or agent execution.
Experience designing evaluations that measure actual task performance and detect regressions or unacceptable failures.
Ability to reason about data permissions, model limitations, reliability, latency, and cost when selecting an implementation.
Experience deploying and observing services in a cloud or containerized environment and explaining results and limitations to users.
Demonstrated use of AI coding tools or agents, with independent review, testing, and security checks of generated code.
Travel up to 50% for user engagement, deployment support, and forward deployed delivery.
Active Secret clearance Required. Please note, only US Citizens can obtain a clearance.
Preferred Qualifications
Experience building AI-driven automation for operational or enterprise workflows with human review and approval steps.
Experience with multiple model providers, open-weight models, model adaptation, or inference optimization.
Experience with Palantir Foundry, Databricks, or comparable platforms used to deploy AI applications.
Experience working within defense or intelligence security constraints, restricted networks, or resource-constrained deployment environments.
The target salary range for this position is $140,000-$200,000.
The salary range displayed represents the typical salary range for this position and is not a guarantee of compensation. Individual salaries are determined by various factors including, but not limited to location, internal equity, business considerations, client contract requirements, and candidate qualifications, such as education, experience, skills, and security clearances.