Consultative Problem Solving
With limited guidance, independently perform deep investigations, identify data issues, and propose solutions that balance performance, cost, risk, and business needs.
Engage business stakeholders to gather ambiguous requirements, ask the right questions, and translate them into clear technical designs.
Provide thought leadership and recommend technical patterns, frameworks, and toolsets.
Data Quality, Reliability & Operations
Implement robust data quality frameworks, monitoring, and alerting to ensure high trust in business-critical data assets.
Troubleshoot data inconsistencies and ensure proper logging, testing, and recovery mechanisms across pipelines.
Lead regression testing, software upgrades, and production deployments with strong change control discipline.
Collaboration & Leadership
Lead all phases of solution development—from design to deployment and operationalization.
Mentor and guide other engineers in coding standards, architecture patterns, Databricks best practices, and AWS platform usage.
Partner with Data Architecture, Analytics, Product, and Business teams to deliver solutions that improve decision-making.
Provide training sessions and documentation to uplift the data engineering maturity across the organization.
Special Projects
Participate in strategic initiatives such as AI readiness, data unification efforts, metadata strategy, and enterprise integration roadmaps.
Drive continuous improvement in engineering frameworks, onboarding workflows, and platform capability.
Required Skills
5+ years of experience in data engineering, data architecture, or large-scale distributed data systems.
Expert-level experience with cloud platforms such as AWS, GCP, or Azure, leveraging services for data storage, ingestion, pipeline orchestration, database or lake house management, data transformation.
5+ years of hands-on experience designing, developing, and supporting enterprise-scale data pipelines on the Databricks Lakehouse platform using PySpark, Delta Lake, Databricks Workflows, and Lakeflow Declarative Pipelines
