Build and manage curated data layers following Lakehouse Medallion architecture best practices (Bronze / Silver / Gold).
Develop reusable, modular data transformation frameworks to accelerate delivery across domains.
Databricks Application Development
Design and develop Databricks‑native data applications, including notebook‑based apps, Databricks dashboards, and interactive data experiences for analytics and business users.
Build data APIs, parameterized pipelines, and app‑integrated data services leveraging Databricks and Lakehouse capabilities.
Partner with analytics, AI, and application teams to embed data and insights directly into workflows and applications.
Ensure Databricks apps meet performance, security, governance, and usability standards.
Performance, Scalability & Reliability
Optimize Apache Spark jobs and Databricks workloads for performance, cost efficiency, scalability, and reliability.
Proactively address challenges related to data volume, schema evolution, and compute optimization.
Implement robust data quality checks, validations, and anomaly detection within pipelines and apps.
Production Support & Operations
Own and support production data pipelines and Databricks applications, including monitoring, troubleshooting, and root‑cause analysis.
Ensure high availability, data correctness, and SLA adherence for business‑critical datasets and apps.
Contribute to observability, alerting, and operational automation.
Full‑Stack Data Enablement
Collaborate with BI, analytics, AI/ML, platform, and application teams to deliver end‑to‑end data solutions.
Enable data consumption across dashboards, reports, Databricks apps, AI models, APIs, and downstream applications.
Translate business and analytical requirements into well‑designed data pipelines and data applications.
Engineering Excellence & Technical Influence
Act as a technical leader and mentor, defining best practices for data engineering and Databricks app development.
Participate in architecture reviews, design discussions, and technical roadmaps.
Continuously evaluate and adopt modern Databricks features, GenAI capabilities, and automation patterns to improve developer productivity.
Required Skills & Experience
5+ years of hands‑on data engineering experience, owning production‑grade pipelines and data solutions.
