We are a fungible mix of front-end back-end and ML/AI engineers driving higher customer conversion through AI-powered onboarding building a suite of AI capabilities into our consoles - LLM agents personalization recommendations and experimentation - and unifying our platform at the architectural level. The harness is where these come together: a mixture-of-experts agent platform on Amazon Bedrock (AgentCore/Strands) MCP servers that give agents persistent working memory and typed access to backend services a RAG-backed knowledge base for conversational Q&A ranking and relevance layers that order recommendations and retrieved context and offline/online evaluation harnesses that measure agent quality and gate every change.
At Amazon we hire the best minds in technology to innovate and build on behalf of our customers. As a Senior SDE you'll lead the design and delivery of major components of the harness - with a particular emphasis on ranking and evaluation - tackle ambiguous greenfield problems where no playbook exists and raise the engineering bar across the team through design review mentorship and operational excellence. Because the harness is a multi-product platform you'll partner closely with other engineers UX and applied science across products - your work shows up in customer conversion admin productivity and the velocity of every team building on the platform.
Key job responsibilities
Lead the ranking and evaluation systems for the harness - relevance and ranking of recommendations and retrieved context and the offline/online evaluation harnesses (golden datasets LLM-as-judge regression and drift detection human-in-the-loop feedback) that measure agent quality and gate changes before they reach customers. -Design the metrics and experimentation that turn agent quality into decision-grade signal (task completion recommendation acceptance conversion) and close the loop from production feedback back into ranking and model/prompt improvements. -Lead the design and delivery of other major harness components - the agent loop orchestration intent routing session and memory tool/function calling or streaming - and contribute significantly to its multi-quarter technical roadmap. -Help define the platform contracts (MCP tool interfaces identity pass-through agent-to-service seams) that let specialized expert agents and product teams build on the harness safely and independently. -Design and scale parts of a mixture-of-experts agent architecture on Amazon Bedrock (AgentCore/Strands) that is low-cost to train and serve with clear patterns for adding new domains. -Contribute to safety and relevance guardrails and the production quality/safety bar alongside the evaluation systems above. -Build highly scalable low-latency fault-tolerant distributed systems and share in operational excellence for the harness (latency availability cost on-call readiness). -Raise the technical bar across the team: participate in and lead design reviews mentor SDEs and influence direction with UX applied science and partner teams across products.
A day in the life
You might spend the morning extending the evaluation harness with new golden cases and a regression gate for the recommendation agent pair with an applied scientist at midday to tune the ranking layer and read out an A/B result and in the afternoon shape the MCP contract for a new expert agent - then write the one-pager that aligns the teams building on it. You'll operate in an agile environment where development cycles are measured in weeks take on the hardest and most ambiguous parts of the platform and be one of the technical leaders the team turns to when the answer isn't obvious. WorkSpaces Forefront owns the admin experience across the WorkSpaces Consoles (Personal Applications) and the WorkSpaces SDK. We're building an agentic control plane for Console and programmatic experiences - an agentic harness (LLM agents routing ranking personalization memory knowledge bases MCP tools and evaluation) plus experimentation and closed-loop learning - while unifying key infrastructure toward a federated console architecture.
About the team
ABOUT AWS
Diverse Experiences
Amazon values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.
Why AWS
Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Work/Life Balance
We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
Inclusive Team Culture
Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity and AmazeCon conferences, inspire us to never stop embracing our uniqueness.
Mentorship and Career Growth
We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
5+ years of non-internship professional software development experience
5+ years of programming with at least one software programming language experience
5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
Experience as a mentor, tech lead or leading an engineering team
5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
Bachelor's degree in computer science or equivalent
Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
USA, WA, Seattle - 168,100.00 - 227,400.00 USD annually
