Follow the video below to know about us! Clipchamp
Role Overview
We are looking for an experienced Azure Data Engineer with strong hands-on expertise in Azure Databricks, PySpark, SQL, Delta Lake, and Medallion Architecture. This role primarily focuses on production support, code migration, and implementation of released changes within an existing enterprise data platform.
The ideal candidate will be comfortable working with Git repositories, identifying ticket-specific code changes, safely integrating approved updates, troubleshooting production issues, and coordinating with client teams throughout migration and release activities.
Key Responsibilities
1. Production Support & Troubleshooting
Work closely with client and development teams to understand production issues, change requests, and released tickets.
Troubleshoot production pipeline failures, data issues, and transformation errors.
Perform data validation, reconciliation, and root cause analysis.
Investigate issues across existing enterprise data platforms and implement approved fixes.
Coordinate with client teams to ensure timely resolution of production issues.
2. Git Management & Code Migration
Review Git repositories and identify code changes associated with released tickets.
Perform Git cherry-picking, branch management, merge conflict resolution, and code integration.
Apply approved changes to existing Databricks notebooks, PySpark code, SQL transformations, and data pipelines.
Ensure code changes are integrated safely into existing codebases.
Support migration of changes across Development, UAT, and Production environments.
3. Azure Databricks & Data Engineering
Support and maintain data pipelines using Azure Databricks, PySpark, Azure Data Factory, and ADLS Gen2.
Apply changes to existing notebooks, data transformations, and processing workflows.
Work with Delta Lake and Delta Tables, including MERGE operations, incremental processing, schema evolution, Change Data Feed (CDF), and optimization.
Maintain data quality and consistency across processing stages.
Work with Medallion Architecture to support structured data processing and transformation.
4. Release Management & Deployment Support
Support the migration and implementation of approved changes across Dev, UAT, and Production environments.
Coordinate with client teams during releases and production deployments.
Validate deployed changes and investigate post-release issues.
Follow established code integration, migration, and deployment procedures.
Help maintain the reliability and stability of the existing enterprise data platform.
Required Technical Skills & Competencies
Strong hands-on experience with Azure Databricks and PySpark.
Strong SQL and data transformation skills.
Experience with Delta Lake and Delta Tables.
Good understanding of Medallion Architecture.
Experience with Azure Data Factory and ADLS Gen2.
Strong Git experience, particularly cherry-picking, branching, merging, and conflict resolution.
Production support and troubleshooting experience.
Experience working with existing enterprise data platforms and code migration activities.
Ability to investigate data issues, perform reconciliation, and conduct root cause analysis.
Experience implementing released changes and supporting deployments across multiple environments.
Must-Have Skills
Azure Databricks
PySpark
SQL and Data Transformations
Delta Lake / Delta Tables
Medallion Architecture
Azure Data Factory
ADLS Gen2
Git Cherry-Picking
Branch Management and Merge Conflict Resolution
Production Support and Troubleshooting
Code Migration and Integration
Data Validation and Reconciliation
Dev, UAT, and Production Release Support
Good to Have Skills
Azure Event Hubs
Change Data Capture (CDC)
Debezium
Databricks Workflows and Jobs
Unity Catalog
Azure DevOps and CI/CD
Experience with large-scale data platforms
Data pipeline and performance optimization
Desired Attributes
Hands-on approach to production support and data engineering.
Strong troubleshooting and problem-solving skills.
Ability to quickly understand existing codebases and identify ticket-specific changes.
Careful and systematic approach to cherry-picking, code integration, and migration.
Strong understanding of data quality and transformation logic.
Ability to coordinate effectively with client and development teams.
Strong ownership of release activities and production stability.
Attention to detail when implementing changes across environments.
Interview Process
2–3 rounds of discussion.
