Deploy and operate GPU inference services on Kubernetes: autoscaling, containerization and infrastructure as code.
Build CI/CD for model artifacts, with model versioning and reproducible releases.
Own observability and performance benchmarking: p50/p99 latency, real-time factor (RTF), GPU utilization and cost per request.
Partner with research to productionize new models.
What You’ll Bring
Hands-on experience deploying, scaling and optimizing ML models in production, beyond consuming model APIs.
Working knowledge of the full inference stack, from CUDA kernels to serving frameworks, with depth in some layers; expertise across all of them is not expected.
GPU performance optimization: profiling (Nsight Systems, PyTorch Profiler), memory bandwidth, CPU-GPU synchronization.
Solid understanding of model serving: continuous batching, KV cache management, request scheduling.
Production experience with Kubernetes, CI/CD and cloud infrastructure.
Strong Python and PyTorch, and a rigorous approach to performance benchmarking.
Experience with speech or audio models (TTS, ASR, voice conversion).
Experience with inference frameworks such as vLLM (vLLM-Omni), SGLang, TensorRT-LLM, TensorRT or Triton Inference Server.
Compensation
When determining compensation, a number of factors will be considered, including skills, experience, job scope, location, and competitive compensation market data.
Benefits for U.S.-based roles
Competitive salary and generous company equity
Medical, dental, and vision insurance – 99.99% of premiums covered by Cantina
42 days of paid time off, including
15 PTO days
10 sick days
15 company holidays
2 floating holidays
Generous parental leave & fertility support
401(k) retirement savings plan
Lifestyle spending account – $500/month to use however you’d like
Complimentary lunch and snacks for in-office employees
One Medical membership, and more!