Develop and maintain ETL/ELT pipelines that efficiently process large volumes of structured and unstructured data from diverse sources.
Ensure data quality, governance, and security standards are embedded throughout all data engineering processes.
2) Technical Implementation
Hands-on development of Databricks notebooks using PySpark, Python, and SQL for ETL automation.
Create and optimise PySpark scripts for efficient data extraction, transformation, and loading from large datasets.
Implement custom data manipulation, validation, and error handling solutions to enhance ETL robustness.
3) Technical Guidance
Provide technical mentorship to junior data engineers and analysts.
Lead code reviews, establish best practices, and drive adoption of modern data engineering tools and methodologies.
Collaborate with cross-functional teams including data scientists, analysts, and software engineers to deliver integrated solutions.
4) Performance & Optimisation
Monitor and optimise data pipeline performance, implementing solutions for scalability and cost-effectiveness.
Conduct testing, debugging, and troubleshooting of data transformation processes.
Verify data integrity throughout pipeline stages and resolve complex technical issues.
To qualify for the role, you must have
Bachelor's Degree in Computer Science, Engineering, Data Science, or related field OR Master's Degree in Data Engineering, Computer Science, or MBA (preferred)
5-7 years of data engineering experience with AWS-native implementations
Proven experience working as part of a data engineering team on ETL migration projects
Hands-on experience implementing Databricks notebooks using PySpark, Python, and SQL for ETL automation
Demonstrated expertise developing PySpark scripts for efficient data extraction, transformation, and loading from large datasets
Experience utilising Python for custom data manipulation, validation, and error handling in ETL processes
Proficiency employing SQL for complex joins, aggregations, and database operations within Databricks environments
Track record of testing, debugging, and optimising data transformation processes for accuracy and performance
Experience verifying data integrity throughout pipeline stages and resolving troubleshooting issues
Proven ability collaborating with cross-functional teams to align ETL migration tasks with project goals and deadlines
Experience in government sector data projects and compliance requirements (preferred)
Experience in HR domain including workforce analytics, payroll systems, and employee data management (preferred)
Experience providing technical mentorship to junior team members (preferred)
Certified in
AWS Certified Data Engineer - Associate (preferred)
AWS Certified Solutions Architect or AWS Certified Big Data - Specialty (preferred)
Databricks Certified Data Engineer Associate or Professional
Amazon Redshift or Snowflake certifications
Agile/Scrum Master certification (preferred)
Essential Skills
1) Technical Skills
AWS cloud platform expertise (Redshift, S3, Glue, EMR, Kinesis, Lambda)
Databricks platform proficiency with PySpark, Python, and SQL implementation
Data warehouse design and implementation (Amazon Redshift, Snowflake on AWS)
ETL/ELT migration project experience and pipeline development
Real-time streaming technologies (Kinesis Data Streams, Kafka)
Data exchange platforms and API integration (AWS Data Exchange, Partner APIs)
2) Communication Skills
- Clear written and verbal communication of technical concepts to non-technical
- stakeholders
Presentation and reporting abilities for executive audiences
Active listening to understand business requirements
Stakeholder management across technical and business teams
Meeting facilitation for technical design sessions
3) Leadership Skills
Technical mentorship and guidance for junior engineers
Conflict resolution in cross-functional project environments
Decision-making under pressure for critical data system issues
Knowledge transfer and coaching abilities
Change management for data platform migrations
4) Organisational Skills
Time management across multiple concurrent data projects
Resource allocation for optimal pipeline performance
Multi-tasking coordination of data engineering deliverables
Attention to detail in data quality and governance processes
Documentation of technical specifications and system architecture
5) Problem-Solving Skills
Critical thinking for complex AWS data architecture challenges
Creative solution development for scalability and performance issues
Issue escalation and resolution for production data systems
Adaptability to evolving business and technical requirements
Strategic planning for AWS data infrastructure roadmaps
6) Interpersonal Skills
Relationship building with business stakeholders and technical teams
Negotiation of technical requirements and project timelines
Emotional intelligence in managing diverse project teams
Cultural awareness in global data initiatives
Customer service orientation for internal data consumers
What we look for
Highly motivated individuals with excellent problem-solving skills and the ability to prioritize shifting workloads in a rapidly changing industry. An effective communicator, you’ll be a confident team player that collaborates with people from various teams while looking to develop your career in a dynamic organization.
What working at EY offers
EY offers a competitive remuneration package where you’ll be rewarded for your individual and team performance. We are committed to being an inclusive employer and are happy to consider flexible working arrangements. Plus, we offer:
Continuous learning: You’ll develop the mindset and skills to navigate whatever comes next.
Success as defined by you: We’ll provide the tools and flexibility, so you can make a meaningful impact, your way.
Transformative leadership: We’ll give you the insights, coaching and confidence to be the leader the world needs.
Diverse and inclusive culture: You’ll be embraced for who you are and empowered to use your voice to help others find theirs.
Company description
EY is building a better working world by creating new value for clients, people, society and the planet, while building trust in capital markets.
Enabled by data, AI and advanced technology, EY teams help clients shape the future with confidence and develop answers for the most pressing issues of today and tomorrow.
EY teams work across a full spectrum of services in assurance, consulting, tax, strategy and transactions. Fueled by sector insights, a globally connected, multi-disciplinary network and diverse ecosystem partners, EY teams can provide services in more than 150 countries and territories.
All in to shape the future with confidence.
Apply now »
Apply now
Start applying with LinkedIn
Apply Now
Start
Please wait...
Find similar jobs
- Consulting DE,
- Consulting CN,
- Consulting CA,
- Consulting BR,
- Consulting
Connect with us
Our locations
My EY
Site map
Legal and privacy
Labor condition applications
Cookie Consent Manager
Opens in a new tab.
Opens in a new tab.
Opens in a new tab.
Opens in a new tab.
Opens in a new tab.
EY refers to the global organization, and may refer to one or more, of the member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients.
×
Cookie Consent Manager
When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. Because we respect your right to privacy, you can choose not to allow some types of cookies. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.
Required Cookies
These cookies are required to use this website and can't be turned off.
Required Cookies
Show More Details
Required Cookies
- Provider
- Description
- Enabled
SAP as service provider
We use the following session cookies, which are all required to enable the website to function:
"route" is used for session stickiness
"careerSiteCompanyId" is used to send the request to the correct data center
"JSESSIONID" is placed on the visitor's device during the session so the server can identify the visitor
"Load balancer cookie" (actual cookie name may vary) prevents a visitor from bouncing from one instance to another
Cookies from provider SAPasserviceprovider are required and cannot be turned off
Advertising Cookies
These cookies serve ads that are relevant to your interests. You may freely choose to accept or decline these cookies at any time. Note that certain functionality that these third parties make available may be impacted if you do not accept these cookies.
Consent to all Advertising Cookies
Show More Details
Advertising Cookies
- Provider
- Description
- Enabled
AddThis
Google Analytics is a web analytics service offered by Google that tracks and reports website traffic.
Cookie Information
Privacy Policy
Terms and Conditions
Consent to cookies from provider AddThis
LinkedIn
LinkedIn is an employment-oriented social networking service. We use the Apply with LinkedIn feature to allow you to apply for jobs using your LinkedIn profile. Opting out of LinkedIn cookies will disable your ability to use Apply with LinkedIn.
Cookie Policy
Cookie Table
Privacy Policy
Terms and Conditions
Consent to cookies from provider LinkedIn
Google Analytics
Google Analytics is a web analytics service offered by Google that tracks and reports website traffic.
Cookie Information
Privacy Policy
Terms and Conditions
Consent to cookies from provider GoogleAnalytics
Google Tag Manager
Google Tag Manager is a tag management system for conversion tracking, site analytics, remarketing, and more.
Privacy Policy
Terms and Conditions
Consent to cookies from provider GoogleTagManager
Confirm My Choices
- Reject All Cookies
- Accept All Cookies
