Own the enterprise data warehouse as a production system, including data model integrity, pipeline reliability, ingestion service levels, and incident response.
Maintain and improve warehouse architecture, feed logic, and Gold-layer harmonization rules.
Ensure critical platform knowledge is documented and shared across the team.
Improve platform reliability, processing latency, scalability, and cost efficiency across all medallion layers.
Lead, develop, and retain Senior Data Engineers and Data Engineers, establishing clear ownership of critical platform capabilities.
Build a collaborative team through coaching, mentoring, individual development plans, and performance management.
Coordinate individual contributor hiring decisions with the Senior Director and define role profiles and hiring criteria.
Organize competing priorities and provide calm, clear leadership through production issues and changing demands.
Reinforce engineering discipline, documentation, accountability, and shared ownership of reliability.
Partner with Analytics Engineering on Gold-layer table design and Silver-to-Gold promotion logic supporting metric definitions and data products.
Work with QA to embed validation checkpoints and support Tier 1 and Tier 2 pipeline test suite development.
Collaborate with Data Governance on lineage, metadata, and data quality policy requirements.
Support Partner Data Services initiatives requiring new or modified pipelines.
Communicate engineering priorities, dependencies, and technical decisions effectively across peer functions.
Qualifications
Required
Seven or more years of progressive data engineering experience, including at least two years in people management or a team lead capacity. Qualified Team Leads and Tech Leads may be considered.
Deep, hands-on Databricks expertise, including medallion architecture, Delta Lake, Unity Catalog, and production pipeline development and operations.
Strong enterprise data warehouse expertise, including production design, operations, data modeling, and transformation logic.
Proficiency in PySpark, SQL, and Python sufficient to review code and architecture and mentor engineers.
Proven experience with Azure Databricks, Azure Data Lake Storage, and Azure Event Hubs.
Strong Kafka/Event Hubs experience and familiarity with Confluent Cloud.
A servant-leadership approach demonstrated through coaching, team development, and support for engineers.
Strong organization, prioritization, and communication skills, with the ability to manage multiple workstreams and remain composed under pressure.
Effective collaboration with functions such as QA, Data Governance, and Informatics.
Preferred
Healthcare, specialty pharmacy, or regulated-industry experience involving clinical or operational data.
Production experience with Structured Streaming and event-driven ingestion using Azure Event Hubs
