Roles & Responsibilities.
Design and Implement Data Pipelines
Develop and maintain robust data pipelines for ingesting, processing, and storing data from
various sources.
Azure Services Expertise
Utilize Azure services like Azure Data Factory, Azure Synapse Analytics, Azure Databricks,
Azure Data Lake Storage, and Azure SQL Database.
Data Modeling and Architecture
Design and implement data models, ensuring data quality, consistency, and performance.
ETL Processes
Develop and optimize ETL (Extract, Transform, Load) processes using tools like Azure Data
Factory or similar.
Programming Skills
Proficiency in Python, SQL, and potentially Scala or R for data manipulation and processing.
Big Data Technologies
Experience with big data technologies such as Spark, Hadoop, or similar.
Cloud Infrastructure
Understanding of Azure cloud infrastructure services like networking, storage, and security.
Collaboration and Communication
Work effectively with cross-functional teams, including data scientists, analysts, and other
engineers.
Continuous Improvement
Identify opportunities to optimize and improve existing data pipelines and processes.
Mentorship
Guide and mentor junior data engineers.
Key Technical Skills Required
5+ years of experience as a data engineer or similar role, with a focus on Azure.
Proficiency in Azure Data Factory, Azure Synapse, and Azure Databricks.
Strong programming skills in Python and SQL.
Show the full description
Experience with big data technologies like Spark.
Experience with data modeling and ETL processes.
Understanding of cloud infrastructure and security best practices.
Excellent communication and collaboration skills.