You will help determine the most appropriate execution approach for each workload—from SQL-based data processing and native Python to distributed Spark processing and optimized CPU/GPU execution (e.g., Polars, Ray) approaches, including ONNX-based models and technologies such as nvForest. You will act as a critical bridge between our ML Platform and the engineers behind our core runtime—rAVE, our in-house real-time inference runtime. A key part of this role is developing approaches that reduce duplicated computation on the ML side, package models efficiently, and establish strict contracts so models run predictably when handed off to the inference environment.
About the Senior ML Systems Engineer Role
Optimize feature extraction and curation: Identify and implement efficient approaches to extracting, transforming, joining, and curating features for model development and production. Optimize data-processing steps and their dependencies to reduce unnecessary computation, data movement, and infrastructure costs
Build ML productionization frameworks: Develop reusable Python libraries, tools, and workflows that help Data Scientists move models from experimentation into production with minimal rework
Optimize model training pipelines: reduce cloud compute overhead through distributed training, GPU training for neural-network models, data caching, and efficient resource allocation. Additionally, you will build automated monitoring systems to proactively detect and mitigate feature drift, establishing robust continuous training (CT) triggers that keep our production models highly accurate as real-world data evolves
Optimize execution strategies: Evaluate and implement appropriate execution approaches, including native Python, optimized data-processing libraries, Spark, and specialized CPU/GPU formats (e.g., ONNX), avoiding unnecessary infrastructure and runtime overhead.
Optimize models for inference: Improve execution efficiency and lower costs prior to deployment by reducing duplicated computation across models and preparing optimized CPU/GPU formats (e.g., ONNX, nvForest) for runtime ingestion
Integrate with real-time inference: Productionize models for seamless integration with rAVE or other real-time inference components, establishing strict payload contracts and integration patterns during the engineering handoff
Automate the ML lifecycle: Improve model validation, packaging, versioning, testing, deployment workflows, and integration with experiment tracking and model-management tools
Improve performance and reliability: Profile workloads to identify bottlenecks, optimizing model configurations and computational requirements before handing them off to the inference environment.
Enable Data Scientists: Provide production-ready tooling, documentation, and reusable components that simplify the transition from research to production artifacts
Collaborate across engineering teams: Work with Data Scientists, ML Engineers, and platform engineers to deliver production solutions
About you as a Senior ML Systems Engineer
Strong Python engineering skills, including experience building reusable libraries and maintainable production software
Experience productionizing machine-learning models or data-processing workloads beyond the experimentation stage
Understanding of the ML lifecycle, including experiment tracking, model versioning, validation, packaging, and deployment
Experience optimizing computationally intensive workloads, with the ability to profile performance and identify CPU, memory, data-processing, and execution bottlenecks
Experience designing automated testing and CI/CD workflows for data or ML systems
Understanding of the trade-offs between local or native Python execution, distributed processing, and infrastructure-heavy approaches
Familiarity with Spark or PySpark (and alternatives such as Polars or Ray)and experience working with large-scale data-processing pipelines
Familiarity with Rust or experience working across language boundaries, with the ability to establish strict contracts and understand architectural integration with a Rust-based inference runtime(s)
Ability to collaborate effectively with Data Scientists and software engineers, translating ML requirements into production-quality implementations.
Evaluate alternative execution engines: Benchmark workloads on Polars and Ray (on GKE) against Spark/Databricks for cost and performance
Modernize development workflows: Support the move from notebooks to IDE-native development (e.g., VS Code), with packaged, testable code
Technologies and Experience Core technologies
Python and its data ecosystem, including NumPy, pandas, and PyArrow
MLflow and ML experiment-management workflows.
Databricks and Spark/PySpark
Containerization (Docker, GKE/Kubernetes), CI/CD, workflow orchestration (Airflow), and cloud infrastructure, preferably GCP (GCS)
Model packaging, automated testing, and production monitoring
Additional experience that is a plus
SQL and experience working with OLAP and/or OLTP systems
Understanding of analytical data stores, transactional data systems, and the trade-offs involved in extracting and serving data for ML workloads
ONNX and ONNX Runtime, GPU inference, neural-network training (PyTorch), or neural-network deployment
Rust or experience integrating components implemented in different programming languages
Bigtable, Aerospike, or similar low-latency data stores.
Feature stores, online feature serving, and real-time data pipelines.
Experience with advertising technology, bidding systems, or other latency-sensitive environments
What Success Looks Like
Data Scientists can move models from experimentation to production through reusable, well-documented tooling
Production workloads use execution approaches appropriate to their performance and cost requirements, avoiding unnecessary Spark or infrastructure overhead
Model packaging, validation, and deployment become more automated, consistent, and reproducible
CPU and GPU inference workloads demonstrate measurable improvements in performance, resource utilization, or cost.
Shared-computation opportunities are identified and implemented where they provide measurable benefits
Productionized models integrate reliably with rAVE and the wider ML Platform through predictable and strict engineering contracts.
DT (NASDAQ: APPS) is a mobile growth platform that powers superior experiences and outcomes for advertisers, app developers, carriers, OEMs, and publishers worldwide.
Our technology connects brands and apps with consumers across devices, apps, and mobile environments — helping partners drive awareness, acquisition, engagement, and monetization through intelligent, integrated experiences.
Headquartered in Austin, Texas, DT operates globally across North America, EMEA, APAC, and LATAM.
DT is an equal opportunity employer committed to fostering an inclusive and diverse workplace. We welcome people of different backgrounds, experiences, abilities, and perspectives, and believe teams with different points of view build better products, ideas, and experiences.