Responsibilities
Design, build, and maintain scalable batch and streaming data pipelines that feed our data warehouse and data lake
Understand the advantages and disadvantages of existing and proposed design solutions
Provide day-to-day technical leadership to a team of data engineers, including story definition, task planning, code review, and delivery oversight
Implement and follow data modeling, integration, and lifecycle standards established by the team, and contribute to refining them
Own the implementation and ongoing operation of our Databricks Cloud Data Warehouse, optimizing for performance, cost, scalability, and maintainability
Build data quality checks, lineage tracking, and metadata practices into pipelines from the start
Implement data governance controls (access, retention, accuracy, and integrity) in partnership with the governance and security teams
Apply security and privacy requirements, such as HIPAA and SOX, in pipeline and platform design as appropriate
Partner with data analysts and business stakeholders to understand data needs and explain technical tradeoffs clearly
Monitor, troubleshoot, and optimize data workflows, and lead root-cause analysis on production issues
Mentor and coach data engineers on engineering best practices, testing, CI/CD, and emerging tools
Additional responsibilities as assigned to help drive our mission of improving the lives of everyone living with cancer.
Required Qualifications
Highschool diploma or GED
7+ years of hands-on experience in data engineering or the Big Data domain.
2+ years leading or mentoring engineers in a technical lead or senior engineer role.
Strong hands-on experience with modern Big Data technologies and cloud data warehouses (Databricks preferred, Snowflake, or Fabric is acceptable).
Solid experience building scalable data solutions on a leading cloud platform (Azure preferred)
Proficiency in SQL and Python (or another OO Language).
Preferred
Experience in orchestration and transformation tools (e.g., Airflow, Azure Data Factory)
Working knowledge of data modeling methodologies (Dimensional, 3NF, Data Vault) and experience applying them in warehouse and data lake implementations.
Experience implementing data governance, security, and compliance requirements in data pipelines.
Essential Competencies
Strong communication skills, with the ability to explain technical challenges and solutions clearly to varied stakeholders
Proactive, productive, and analytical problem solver with a track record of troubleshooting and optimizing complex data workflows
Ability to break down projects into discrete tasks/components and create user stories for them
Attendance is an essential job function
