Responsibilities
Define and own the technical direction of the LLM serving deployment, including engine integration, continuous batching, KV cache management and reuse, and quantized execution
Partner with inference performance engineering teams, with accountability for time-to-first-token, inter-token latency, throughput per GPU, and P95/P99 tail latency
Define distributed inferencing strategy, including disaggregated prefill/decode, tensor and pipeline parallelism, KV cache offload across GPU memory, host memory, and RDMA-attached storage
Evaluate emerging runtimes, quantization schemes, speculative decoding, and mixture-of-experts serving, and determine whether each runtime is adopted, developed in-house, or declined
Define the orchestration layer supporting the runtime, including model admission, GPU scheduling and partitioning, cache-aware request routing, and autoscaling
Mentor engineers, lead design and architecture reviews, and present technical direction to business unit and executive audiences
Knowledge and Skills
Required
Production experience with LLM inference engines such as vLLM, SGLang, TensorRT-LLM, TGI, or NVIDIA NIM, including modification of engine internals
Comprehensive understanding of inference internals, including continuous batching, paged attention, KV cache reuse and prefix caching, chunked prefill, quantization, and speculative decoding
Tensor and pipeline parallelism, NCCL collective operations, and the GPU memory hierarchy and interconnect characteristics that govern them
Expert level proficiency in Kubernetes platform architectures, including operators, custom resources, controllers, and scheduling
Strong programming proficiency in Go and Python, with the ability to read, debug, and profile C++/CUDA using tools such as Nsight
Experience with debugging/profiling multi-tier application workloads such as RAG, Agents, etc
Excellent analytical, debugging, and problem-solving abilities
Preferred
Upstream contribution to vLLM, SGLang, TensorRT-LLM, LLM-D, LMCache, or KServe
Disaggregated prefill/decode serving, or KV cache offload and reuse at scale
RDMA, GPUDirect Storage, InfiniBand, or RoCE
MIG, fractional GPU allocation, and multi-tenant GPU isolation
On-premises, air-gapped, or regulated enterprise software delivery
Experience and Education
Minimum of 12 years of experience in Software Engineering, including +1 years working directly on LLM inference runtimes or production model serving
Degree in Computer Science or related field
Accessibility
HPE is committed to creating an inclusive and accessible workplace and encourages applications from all qualified individuals, including those with disabilities. If you believe you require accommodation during any stage of the application or interview process, please submit your request by completing our secure form linked here.
Note: This option is reserved for applicants needing assistance/reasonable accommodation related to a disability.
What We Can Offer You
Health & Wellbeing
We strive to provide our team members and their loved ones with a comprehensive suite of benefits that supports their physical, financial and emotional wellbeing.
Personal & Professional Development
We also invest in your career because the better you are, the better we all are. We have specific programs catered to helping you reach any career goals you have — whether you want to become a knowledge expert in your field or apply your skills to another division.
Unconditional Inclusion
We are unconditionally inclusive in the way we work and celebrate individual uniqueness. We know varied backgrounds are valued and succeed here. We have the flexibility to manage our work and personal needs. We make bold moves, together, and are a force for good.
Let's Stay Connected
Follow @HPECareers on Instagram to see the latest on people, culture and tech at HPE.
#unitedstates
Job
- Engineering
- Job Level:
- TCP_05
- "The expected salary/wage range for this position is provided below. Actual offer may vary from this range based upon geographic location, work experience, education/training, and/or skill level.
- United States of America: Annual Salary USD 165,500 - 313,500 in Colorado // 157,000 - 361,000 in North Carolina & Texas
- The listed salary range reflects base salary. Variable incentives may also be offered."
Information about employee benefits offered in the US can be found at https://myhperewards.com/main/new-hire-enrollment.html
The estimated job application period closure is January 11 2027; this timeline is provided for transparency and internal planning purposes.
HPE is an Equal Employment Opportunity/ Veterans/Disabled/LGBT employer. We do not discriminate on the basis of race, gender, or any other protected category, and all decisions we make are made on the basis of qualifications, merit, and business need. Our goal is to be one global team that is representative of our customers, in an inclusive environment where we can continue to innovate and grow together. Please click here: Equal Employment Opportunity.
Hewlett Packard Enterprise is EEO Protected Veteran/ Individual with Disabilities.
HPE will comply with all applicable laws related to employer use of arrest and conviction records, including laws requiring employers to consider for employment qualified applicants with criminal histories.
Recruitment Fraud Alert
We have become aware of an increase in fraudulent recruitment activities in which individuals impersonate our company or authorized recruitment agencies to offer fake employment opportunities. These scams may occur through false websites, emails, social media, or chat-based applications and often aim to obtain personal information or money. Please note that Hewlett Packard Enterprise (HPE), its direct and indirect subsidiaries and affiliated companies, and its authorized recruitment agencies/vendors will never charge a candidate a registration fee, hiring fee, or any other fee in connection with its recruitment and hiring process. We also never request personal information such as back account details, Social Security numbers, or national IDs via social media or chat applications.
All legitimate job opportunities will come through official company channels, and candidates are responsible for verifying the credentials of any third party claiming to represent the company. Any reliance on fraudulent communication is at the individual’s own risk, and HPE disclaims legal liability for any resulting damages. If you suspect recruitment fraud, do not share personal information or make any payments and report the incident to your local authorities immediately.
