Skills and competencies
Own and continuously improve the CI/CD pipeline every agent and every model or prompt change moves through, from initial commit through sandbox, development, and staging, before anything reaches production.
Design and maintain the automated gates that decide whether a release is safe to promote — evaluation score thresholds, security scan results, guardrail test results — so a human isn't manually eyeballing test output before every promotion.
Own progressive rollout for model and agent changes: canary a change to a small slice of traffic or sessions first, watch the quality and cost metrics live, and only promote to full traffic once those metrics hold.
Build and maintain the rollback path for every stage of the pipeline, so that a bad release can be reversed in minutes, not hours, and so the trigger conditions for rollback are automatic rather than someone noticing a problem by chance.
Own model tiering across the agent fleet: routing agents to the model tier that actually fits their task complexity, and adjusting that routing as usage patterns or model pricing change.
Manage the shadow-period process for newly promoted agents, making sure the right metrics and review checkpoints are actually in place and visible before an agent moves from shadow to fully autonomous operation.
Run model version regression testing whenever a new model version becomes available, confirming existing agents don't silently degrade before that version gets adopted anywhere in production.
Partner with the Agent Runtime Engineer on the runtime side of releases to quickly troubleshoot issues in case of a bad rollout.
Must-Have
At least 5 years in CI/CD, release engineering, or MLOps, including 1–2 years releasing LLM or agent-based systems specifically.
Has designed and run a progressive rollout strategy (canary, shadow, or blue-green) for something stateful or model-driven, with automatic rollback triggers built in from the start, not a person noticing a problem and rolling back by hand.
Has gated a release on evaluation scores before, treating a regression as a hard stop the same way a failed test would, and can design that same kind of check to catch a new model version quietly degrading an existing agent before it touches production.
Strong Python (or similar) and solid infrastructure-as-code skills. Terraform, CDK, GitHub Actions, or CodePipeline all count.
Comfortable building and deploying containerized services with Docker, and owning their release path through CI/CD end to end.
Strong Advantage
Direct experience with AWS Bedrock, or something comparable, including its deployment and versioning quirks.
Has made model tiering or cost-routing decisions — matched task complexity to model cost instead of defaulting everyone to the priciest option.
Worked somewhere release pipelines needed to produce evidence for an audit, not just pass internally.
Can explain the difference between traditional CI/CD (pass or fail) and AI release gating (thresholds and probability) to someone who's only worked in the former. That gap trips people up more than it should.
#LI-CV1
Netskope is committed to implementing equal employment opportunities for all employees and applicants for employment. Netskope does not discriminate in employment opportunities or practices based on religion, race, color, sex, marital or veteran statues, age, national origin, ancestry, physical or mental disability, medical condition, sexual orientation, gender identity/expression, genetic information, pregnancy (including childbirth, lactation and related medical conditions), or any other characteristic protected by the laws or regulations of any jurisdiction in which we operate.
Netskope respects your privacy and is committed to protecting the personal information you share with us, please refer to Netskope's Privacy Policy for more details.
The application window for this position is expected to close within 50 days. You may apply by filling out the below information, or visiting our Netskope Careers site.
