Key job responsibilities
Design and develop large-scale data structures and pipelines that support analytics, machine learning, and business intelligence across the organization
Implement data ingestion solutions using both real-time and batch processing approaches, applying best practices in data modeling and ETL/ELT processes with AWS technologies and big data tools
Gather and translate business requirements into scalable, maintainable technical solutions that integrate seamlessly with our overall data architecture
Analyze source data systems, document findings, and collaborate with source teams to establish and share best practices
Mentor and support junior data engineers by reviewing their work, providing guidance on technical decisions, and helping them grow their skills
Build governed gold datasets and datamarts for ASCS programs, designed for AI use
Move deployments to versioned, tested releases through continuous integration and delivery (CI/CD).
Own pipeline health, run times and query performance, with AI-driven checks for data drift and stale data.
Lower infrastructure (IMR) cost by retiring unused assets
A day in the life
As a Data Engineer II, you'll spend your time architecting and building large-scale data solutions that support analytics and insights across ASCS. You might start your day reviewing business requirements from stakeholders and the BIE team, then transition into designing data pipelines that ingest information from catalog, listing, FCs and offer data, and design them so both stakeholders and generative AI (GenAI) tools like ANDI can use them. You'll collaborate with BIEs and SDEs to evaluate different technical approaches, write and test code using tools like AWS Redshift, Spark, and Hive, and document your solutions so others can understand and build upon your work. Beyond your core responsibilities, you'll have opportunities to guide junior engineers through complex problems and contribute to discussions about which tools and technologies best serve our evolving needs.
About the team
You'll join CATALIST (Catalog Analytics, Insights, Strategy and Tech), the central Business Intelligence Engineering (BIE) team in Catalog Support Programs (CSO), within ASCS. CSO runs program management, technical support and strategic initiatives that improve the customer experience. CATALIST builds self-service analytics and AI-powered insights for programs such as BLMA, Package Hierarchy (PH), World Product Catalog (WPC)
3+ years of data engineering experience
4+ years of SQL experience
Experience with data modeling, warehousing and building ETL pipelines
Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.