Role Overview
You keep the inference platform running. You build the monitoring, alerting, capacity dashboards and incident handling, and the automation that lets a small team support production models reliably.
What will you do in this role
Build observability for the inference layer, from GPU to gateway: GPU health and utilisation (DCGM, including XID and ECC errors), and engine metrics such as time to first token, inter-token latency, queue depth, KV cache use, preemptions, error rates and token throughput.
Run synthetic end-to-end probes that check the content of model responses, not only the HTTP status, since some serving failures return HTTP 200 with empty output.
Create the capacity and status dashboards that show what runs where and how much capacity is free.
Implement the SLOs, error budgets and alerts that the lead defines; run on-call and incident response; facilitate post-incident reviews.
Handle incidents, problems and changes through IT service management (ServiceNow or similar).
Automate operations: health checks, self-healing, safe upgrades, and the signals that tell the gateway when to spill traffic over to cloud models. Coordinate node draining and maintenance with the hardware and cluster team.
Run capacity and resilience tests and keep runbooks and the support model up to date.
Work with the hardware and cluster team on hardware faults and cluster issues, and with security on access and logging.
What should you have
Bachelor's degree in information technology, Computer Science or related field
3 to 6 years of hands-on experience working in full-stack development
Required Skills
DevOps or platform engineer running production services.
Hands-on Prometheus and Grafana (or equivalent), alerting and on-call practices.
Kubernetes and Linux operations; scripting in Python or Bash.
Familiarity with LLM serving engines (vLLM, SGLang or Triton) and their metrics.
Infrastructure as code and CI/CD.
Primary skills (must have)
Site Reliability Engineering (SRE) / DevOps / Platform Engineering
Prometheus & Grafana (or equivalent monitoring and observability tools)
Monitoring, Alerting and Incident Management practices.
Kubernetes Administration & Operations.
Linux System Administration / Operations
Python or Bash Scripting for automation and operational tasks.
LLM Serving Engines / Inference Infrastructure such as vLLM, SGLang, or Triton
Infrastructure as Code (IaC).
CI/CD Pipelines and Deployment Automation
Secondary skills (Nice to have)
Experience operating GPU clusters (NVIDIA DCGM, GPU Operator) or HPC environments.
Experience with ServiceNow incident, problem and change processes.
Cybersecurity: security monitoring and logging for production platforms, Kubernetes security (RBAC, network policies, admission control), vulnerability management, and handling incidents together with a security operations team.
Who we are
We are known as Merck & Co., Inc., Rahway, New Jersey, USA in the United States and Canada and MSD everywhere else. For more than a century, we have been bringing forward medicines and vaccines for many of the world's most challenging diseases. Today, our company continues to be at the forefront of research to deliver innovative health solutions and advance the prevention and treatment of diseases that threaten people and animals around the world.
What we look for
Imagine getting up in the morning for a job as important as helping to save and improve lives around the world. Here, you have that opportunity. You can put your empathy, creativity, digital mastery, or scientific genius to work in collaboration with a diverse group of colleagues who pursue and bring hope to countless people who are battling some of the most challenging diseases of our time. Our team is constantly evolving, so if you are among the intellectually curious, join us-and start making your impact today.
Additional location(s) Compensation Information
Required Skills
Bash (Scripting Language), CI/CD, Deployment Automation, DevOps, Grafana, Infrastructure As Code (IaC), Kubernetes, Large Language Model (LLM) Fine-Tuning, Linux System Administration, Platform Engineering, Prometheus (Software), Python (Programming Language), Site Reliability Engineering Preferred Skills:
Current Employees apply HERE
Current Contingent Workers apply HERE
Secondary Language(s) Job Description
#MSDHYDIT
Search Firm Representatives Please Read Carefully
Merck & Co., Inc., Rahway, NJ, USA, also known as Merck Sharp & Dohme LLC, Rahway, NJ, USA, does not accept unsolicited assistance from search firms for employment opportunities. All CVs / resumes submitted by search firms to any employee at our company without a valid written search agreement in place for this position will be deemed the sole property of our company. No fee will be paid in the event a candidate is hired by our company as a result of an agency referral where no pre-existing agreement is in place. Where agency agreements are in place, introductions are position specific. Please, no phone calls or emails.
Employee Status
- Regular
- Relocation:
- Domestic
- VISA Sponsorship:
- No
- Travel Requirements:
- No Travel Required
- Flexible Work Arrangements:
- Hybrid
- Shift:
- Not Indicated
- Valid Driving License:
- No
- Hazardous Material(s):
- n/a
- Job Posting End Date:
- 11/24/2026
- A job posting is effective until 11:59:59PM on the day BEFOREthe listed job posting end date. Please ensure you apply to a job posting no later than the day BEFORE the job posting end date.
Requisition ID:R427210
Who can apply
Only those candidates can apply who
have minimum 3 years of experience
Salary
Competitive salary
Experience
3 year(s)
Deadline
2035-01-01 00:00:00