Applies reuse-first, AI-assisted practices to strengthen SDLC-quality routines for data pipelines (e.g., test generation and control validation), ensuring traceability/auditability and alignment to resiliency and security expectations.
Support AI and machine learning initiatives by preparing and engineering data for model development and deployment.
Monitor, troubleshoot, and optimize data workflows for performance and reliability.
Stay current with industry best practices and emerging technologies in data engineering and AI.
Required qualifications, capabilities, and skills
Formal training or certification on data engineering concepts and 3+ years applied experience
Proven experience in the full lifecycle of data engineering, from ingestion to production.
Demonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows with strong validation habits and awareness of data sensitivity.
Ability to review and validate AI-assisted outputs (e.g., query suggestions, test ideas, or model change summaries) before use, escalating when uncertain and following data handling requirements.
Strong hands-on expertise with Databricks, Apache Spark, AWS (S3, Redshift, Lambda, etc.), Airflow, and Kafka.
Proficient in Python programming and SQL for data manipulation and analysis.
Experience with data modeling, ETL development, and data warehousing.
Familiarity with AI/ML workflows and supporting data pipelines for machine learning.
Certifications in relevant technologies (AWS, Databricks, Spark, etc.) are highly desirable.
Excellent problem-solving skills and attention to detail.
Strong communication and teamwork abilities.
Preferred qualifications, capabilities, and skills
Experience with containerization (Docker, Kubernetes).
Knowledge of data governance and compliance standards.
Exposure to big data tools and frameworks.
Ability to work in a fast-paced, agile environment.
