What You’ll Be Doing
Play a key role in shaping the future of our data interoperability platform.
Design and operate secure, high-performance data pipelines for real-time and batch public health data flows.
Architect and implement scalable solutions that move sensitive health data reliably and meet strict security and compliance standards.
Build and optimize ETL/ELT pipelines using NiFi, Kafka, Python, and SQL.
Build and maintain CI/CD pipelines for NiFi flows, Python, and SQL using GitHub and GitHub Actions, including flow versioning through NiFi Registry and automated promotion across dev, test, and production.
Deploy and operate data pipelines on Azure and AWS, partnering with DevOps/SRE to integrate them into existing cloud infrastructure and tooling.
Apply CI/CD best practices: pull request reviews, branch protection, automated testing and validation, parameterized environment configuration, and versioned releases with rollback.
Collaborate with interoperability, product, and data science teams to migrate legacy pipelines to modern architectures.
Fine-tune data lakes and ensure smooth data exchange across systems.
Define monitoring and alerting strategies (e.g., Grafana dashboards and alerts) and create runbooks to support operational teams.
Troubleshoot production issues to maintain stable and performant pipelines.
Drive continuous improvement across the data ecosystem through collaboration, innovation, and strong execution.
Contribute directly to building and delivering high-quality, scalable integrations that drive rapid product progress.
- Collaborate closely with teammates to share knowledge and ensure smooth, efficient development workflows.
- What We're Looking For
- Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field.
3+ years of experience implementing enterprise-grade data pipelines, or 7+ years of related professional experience (e.g., software, data, or integration engineering) that includes building and operating production data pipelines.
Hands-on expertise with NiFi, Kafka, Python, and SQL, including designing, operating, and optimizing ETL/ELT pipelines in the cloud.