Tune performance and cost across Spark, Databricks, SQL engines and NoSQL stores. Design secure, resilient solutions on AWS using services such as S3, Glue, EMR, Lambda, Redshift and Athena.
Govern the models, aggregates and data marts behind reporting so that business and regulatory reports are accurate, reconciled, timely and fast.
Drives adoption and governance of approved AI-assisted engineering practices across teams to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test acceleration, release readiness, incident/root-cause analysis), while establishing measurable validation standards (secure coding, peer review, automated testing) and promoting reuse of proven patterns and automation within the SDLC/TLM toolchain.
Applies knowledge of tools within the Software Development Life Cycle toolchain, including approved AI-assisted development and automation capabilities, to improve the value realized by automation at scale.
Enforce modular design, code reviews, CI/CD, automated testing, monitoring and Git-based workflows. Work with application, API and UI teams, using your knowledge of the full software stack to shape integration decisions.
Promote reuse-first practices that use enterprise-authorized AI capabilities for design analysis, test generation, control validation and documentation. Validate all outputs, respect data sensitivity, and keep work traceable and auditable.
Required qualifications, capabilities, and skills
Formal training or certification on software engineering concepts and 5+ years applied experience
Experience in data engineering with a track record of leading teams, mentoring engineers, removing obstacles and being accountable for delivery, platform stability and incident resolution.
Expertise in designing and developing data warehouses and data lakes on Databricks, with strong data modeling skills. Familiarity with Snowflake and comparable platforms such as Redshift, BigQuery or Azure Synapse.
Exceptional SQL skills, including complex queries, optimization and analytical problem solving on large datasets with solid grounding in reporting: business-friendly data layers, BI tools (e.g., Tableau, Power BI, QuickSight), dashboard performance, and accuracy controls.
Outstanding production-grade Python and PySpark, with clean, efficient, testable and maintainable code.
Mastery of data ingestion and processing designs, with a history of building enterprise ETL frameworks and running batch and streaming workloads.
Broad knowledge of relational databases (e.g., PostgreSQL, Oracle), NoSQL stores (e.g., DynamoDB, MongoDB, Cassandra) and AWS data services. You can tune performance across these technologies and choose the right one for each use case.
Demonstrated experience leading effective use of enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security
Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching senior engineers/leads on compliant usage patterns and controls.
A well-rounded understanding of software engineering and application development, including the SDLC, design principles, microservices, APIs (e.g., Spring Boot) and front-end concepts. You also troubleshoot well and know CI/CD and GitHub/Bitbucket workflows.
Experience using enterprise-authorized AI capabilities in engineering work. You review and validate AI-assisted outputs before using them, escalate when unsure, and follow data handling requirements.
Preferred qualifications, capabilities, and skills
Infrastructure provisioning in AWS using Infrastructure as Code (e.g., Terraform, AWS CloudFormation).
Databricks and/or AWS certifications.
Hands-on experience with Snowflake, or with migrating data between platforms.
Experience with orchestration tools (e.g., Airflow) and with data governance and lineage tools.
Experience with containerization (Docker/Kubernetes) and in financial services or other regulated environments.