You will work closely with hardware engineers to understand their debugging and validation workflows, then turn them into automatable workflows. You will also work with cross-functional teams to integrate tools with the production environment. A particular focus will be provisioning, hardware health monitoring, test framework, and data collection. Your work will directly improve Crusoe’s ability to deploy and operate sustainable, AI-first compute systems with world-class performance and reliability.
What You’ll Be Working On
Design, build, and maintain software tools and frameworks that support the hardware development and sustaining lifecycle, including bring-up, validation, deployment, and ongoing production support.
Develop automated test, diagnostics, and health-monitoring frameworks that run across fleets of servers, enabling continuous reliability improvements.
Build tooling, monitoring and logging systems that help hardware engineers triage and root-cause issues across different hardware platforms.
Create automated validation and characterization workflows for GPU, CPU, and high-performance compute platforms, including result collection, analysis, and reporting.
Build and operate data pipelines, dashboards, and analytics that surface hardware failure trends, performance regressions, and reliability metrics.
Support E2E integration and solution testing by developing test harnesses and CI/CD workflows that ensure Crusoe Cloud products meet performance, reliability, and scalability expectations.
Evaluate and introduce new tools, testing methods, and automation approaches that improve engineering efficiency and support Crusoe’s sustainability goals.
Provide data-driven insights from tooling and telemetry to influence Crusoe’s hardware roadmap and reliability strategy.
What You’ll Bring to the Team
5+ years of professional software engineering experience, ideally building internal tools, infrastructure software, provisioning, test automation, or systems software.
Strong proficiency in Python and at least one systems language (Go, C/C++, or Rust), plus Shell scripting.
Experience designing and implementing automation and test frameworks, including workflow engines, for hardware or large-scale infrastructure.
Experience with BMC/Redfish/IPMI, firmware update automation, or out-of-band management.
Experience with fleet management, provisioning, or large-scale automation tools (Kubernetes, Ansible, Terraform, etc.).
Experience building data pipelines, logging, telemetry, or observability tooling to analyze system behavior at scale.
Experience working across hardware, firmware, and software teams in multidisciplinary environments.
Strong analytical and problem-solving skills with a data-driven approach.
Excellent communication and collaboration skills, with the ability to gather requirements from engineers and external partners and translate them into usable tools.
Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience.
Bonus Points
Experience building tooling for GPU clusters, including GPU-to-GPU communication diagnostics (NCCL tests, NVLink monitoring, etc.).
Familiarity with PCIe, InfiniBand, and NVMe/storage concepts and tooling (e.g., lspci, ibstat, nvme-cli, perftest, fio), with the curiosity to go deeper.
Working knowledge of Linux systems, including kernel interfaces, drivers, device tooling, and low-level system diagnostics.
Familiarity with NVIDIA tooling such as DCGM, NVML, and fabric management software.
Experience supporting both ARM and x86 server architectures.
Background in factory test, manufacturing test software, or hardware qualification workflows.
COMPENSATION RANGE
Compensation will be paid in the range of $170,000 - $205,000. Restricted Stock Units are included in all offers. Compensation to be determined by the applicants knowledge, education, and abilities, as well as internal equity and alignment with market data.
Crusoe is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, disability, genetic information, pregnancy, citizenship, marital status, sex/gender, sexual preference/ orientation, gender identity, age, veteran status, national origin, or any other status protected by law or regulation.
