Develop and deploy cloud-based data engineering solutions using modern platforms, distributed processing tools, and containerized environments where applicable.
Build and maintain batch and near real-time data pipelines to meet business, reporting, and application requirements.
Collaborate with cross-functional teams to gather requirements, translate business needs into technical solutions, and support enterprise data initiatives.
Troubleshoot, enhance, and continuously improve data infrastructure, workflows, and related tools to increase scalability, efficiency, and business value.
Remain current with developments in data engineering, cloud platforms, and AI-enabling technologies and apply practical improvements where beneficial.
Comply with applicable ABS Health, Safety, Quality, and Environmental Management System requirements and other internal policies and procedures.
What You Will Need
Education and Experience
Degree in a technical field or equivalent combination of education and experience
8+ years of relevant experience in data engineering or a closely related discipline, including significant experience building and maintaining cloud-based data pipelines and architectures.
Databricks certification (preferred)
Knowledge, Skills, and Abilities
Strong knowledge of data engineering concepts, including data modeling, data integration, ETL/ELT design, and pipeline orchestration.
Strong programming skills in Python and SQL, with the ability to build, test, and maintain production-grade data workflows.
Experience working with cloud-based data platforms such as AWS, Azure, or Google Cloud.
Experience designing, implementing, and supporting data warehouse or Lakehouse architectures such as Databricks, Snowflake, Redshift, or similar technologies.
Knowledge of distributed data processing frameworks and tools, such as Spark or equivalent technologies.
Ability to work with structured, semi-structured, and unstructured data from multiple data sources.
Experience implementing data quality checks, validation frameworks, monitoring, and performance optimization practices.
Knowledge of data governance, lineage, metadata management, and cataloging concepts and tools.
Familiarity with data infrastructure requirements that support AI/ML workflows, including feature engineering, data preparation, and embedding pipelines.
Experience with real-time or streaming data technologies such as Kafka, Kinesis, or similar tools is preferred.
Familiarity with containerization and deployment tools such as Docker, Kubernetes, or similar technologies is preferred.
Ability to analyze technical requirements, solve complex data problems, and deliver practical solutions in a fast-paced environment.
Strong collaboration and communication skills, with the ability to work effectively across technical and business teams.
Ability to manage multiple priorities, meet deadlines, and maintain a high standard of quality and accuracy.
Working knowledge of the ABS Health, Safety, Quality, and Environmental Management System.
Reporting Relationships
Reports directly to the Chief Data Scientist in Global Technology or other management or executive level position.
Working Conditions
Work is primarily sedentary; exerting up to 10 pounds of force occasionally and/or a negligible amount of force frequently or constantly to lift, carry, push, pull or otherwise move object.
Notice: This role has been opened for a future need expected within three months of the original posting date of this position. Potential candidates may not receive communication until the open headcount is confirmed.
