Reporting Line: Lead Data Engineer
Key Responsibilities
Work as part of an agile squad, delivering solutions through agile methodologies.
Collaborate with customers, partners and colleagues to identify and clarify data requirements.
Break down complex problems into smaller, iterative deliverables and contribute to squad backlog prioritisation.
Design, build and enhance the modern data platform, including data pipelines, integrations, data preparation, data models, analytical marts and reporting marts, based on business requirements and agreed design patterns.
Select appropriate tools, methods and design patterns to meet functional and technical requirements.
Proactively identify issues and recommend improvements to platform quality, performance and reliability.
Develop knowledge of preparing and exposing data for use through APIs.
Document, support, manage and maintain the data platform components delivered by the squad.
Provide guidance and training to downstream data consumers on effective use of platform data.
Develop knowledge of supporting and building data APIs.
Promote awareness and understanding of the value and effective use of data across Apex.
Comply with all applicable Apex policies and procedures.
Perform other duties as instructed by the Managing Directors in accordance with Article 6.2 of the employment agreement.
Communication and Collaboration
Work closely with other team members and collaborate with colleagues and stakeholders under the supervision of the Line Manager.
Experience and Skills
At least 2 years of experience in data engineering.
Experience with cloud-based data platforms, such as Snowflake and Databricks.
Experience in data modelling, including dbt.
Experience with data lakes, such as AWS Lake Formation and Azure Data Lake.
Experience with ETL/ELT and data pipeline tools, such as AWS Glue, Azure Data Factory, Fivetran and Stitch.
Experience with data orchestration tools, such as Apache Airflow.
Knowledge of version control systems (such as Git) and CI/CD pipelines.
Experience with messaging, publish/subscribe and queueing systems, such as AWS SNS, AWS SQS, Azure Queue Storage, Kafka and RabbitMQ.
Knowledge of data programming languages, including SQL, Python, Scala or Java.
Experience with cloud workflow services, such as AWS Step Functions, Azure Logic Apps and Camunda.
Experience with interactive query services, such as AWS Athena and Azure Data Lake Analytics.
Experience with event-driven and scheduled processing, such as AWS Lambda and Azure Functions.
Experience with reporting and visualisation tools, such as Power BI, Amazon QuickSight and Mode.
Excellent communication and interpersonal skills.
Strong critical thinking, research and problem-solving skills.
Ability to plan, prioritise and manage your own workload.
Ability to work independently and as part of a wider team.
Working knowledge of agile software development lifecycles.
Disclaimer: Unsolicited CVs sent to Apex (Talent Acquisition Team or Hiring Managers) by recruitment agencies will not be accepted for this position. Apex operates a direct sourcing model and where agency assistance is required, the Talent Acquisition team will engage directly with our exclusive recruitment partners.
