Amazon

DESCRIPTION

As part of the AWS Solutions organization, we have a vision to provide business applications, leveraging Amazon’s unique experience and expertise, that are used by millions of companies worldwide to manage day-to-day operations. We will accomplish this by accelerating our customers’ businesses through delivery of intuitive and differentiated technology solutions that solve enduring business challenges. We blend vision with curiosity and Amazon’s real-world experience to build opinionated, turnkey solutions. Where customers prefer to buy over build, we become their trusted partner with solutions that are no-brainers to buy and easy to use.

Lead our pioneering AI initiative at AWS and define the future of AI agents. Shape how businesses leverage artificial intelligence by creating foundational building blocks used across AWS business applications.

The Core Services team within AWS Applied AI Solutions is creating the foundations that will power the next generation of AI agents from small business to enterprise scale. As a scientific leader, you'll drive our new agentic AI building blocks initiative, pioneering the development of reusable AI components that accelerate and standardize AI product delivery across AWS business applications such as contact center, supply chain, healthcare and life sciences. You'll own one critical capability area such as agent identity and governance, agent collaboration & orchestration, agent evaluation, or knowledge management, and have the privilege to define what we build and how we build it.

This role combines the excitement of a startup environment with the scale of AWS. You'll research state-of-the-art open source and internal tools, tackle highly ambiguous problems, and lead scientific innovation building a V1 product from the ground up. You won't just be implementing someone else's vision — you'll chart the course, define the roadmap, and create novel scientific solutions that eliminate non-differentiating work while ensuring enterprise-grade quality and consistency. If you thrive on ownership, are passionate about AI research and development, and want to fundamentally influence how AWS builds AI products, this role offers an extraordinary opportunity to make your mark.

Key job responsibilities

• Design and implement evaluation frameworks for AI agents, including benchmarking tools, annotation systems for RLHF, and standardized patterns for memory orchestration and retrieval that ensure consistent performance across diverse use cases
• Develop deep expertise in a strategic research area within AI agent systems, becoming the organization's scientific expert in areas such as agent design and evaluation.
• Identify and devise new research solutions for ill-defined customer or business problems that require novel methodologies and paradigms to be invented at the product level
• Articulate key potential scientific challenges of ongoing or future customer needs or business problems in AI agents, and present interventions to address them
• Lead the design, implementation, and successful delivery of solutions for scientifically-complex problems, writing "critical-path" code
• Write clear, useful narratives and documentation describing inventions, solutions, and design choices that enable others to understand and reproduce your work
• Demonstrate detailed knowledge of your team's solutions and systems while proactively driving utilization and improvement upon the state of the art
• Independently assess alternative AI technologies and choose the right approaches to be integrated into your systems
• Apply and drive your team to adopt best practices in scientific research, code development, and technical documentation
• Influence across multiple teams to build consensus on scientific approaches and architectural decisions
• Actively mentor and develop other scientists and engineers, elevating the technical capabilities of the organization

A day in the life

Your morning starts with a team standup, followed by collaborative sessions with software engineers and product managers to understand requirements and internal customer needs. You'll dedicate focused time to researching the latest scientific advances in AI agents and designing novel approaches for your capability area. Throughout the day, you'll work with engineers and scientists within your team to implement solutions, applying scientific rigor to validate approaches. You might lead a scientific design review gathering diverse perspectives to strengthen your methodologies. Your day could include conducting experiments, analyzing results, designing new evaluation methodologies, meeting with internal customers to understand their AI agent needs, or presenting your scientific vision to leadership stakeholders.

About the team

Applied AI Solutions Core Services is a comprehensive suite of core computing services and solutions provided by Amazon Web Services (AWS) to support the foundational infrastructure and operations needs of customers. It encompasses a wide range of essential services and capabilities across AWS. The Core Services team focuses on continuously improving the security, availability, and cost-effectiveness of these critical enterprise-wide services for AWS customers.

Why AWS

Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.

Diverse Experiences

Amazon values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.

Work/Life Balance

We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.

Inclusive Team Culture

Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness.

Mentorship and Career Growth

We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.

BASIC QUALIFICATIONS

• 3+ years of building machine learning models for business application experience
• PhD, or Master's degree and 6+ years of applied research experience
• Experience programming in Java, C++, Python or related language
• Experience with neural deep learning methods and machine learning
• Experience using managed ML/AI solutions
• Deep expertise in at least one AI/ML discipline such as NLP, reinforcement learning, etc.
• Experience with neural deep learning methods and popular frameworks such as PyTorch, TensorFlow, or MxNet
• Experience designing and implementing AI/ML systems, including working with LLMs, prompt engineering, retrieval augmented generation (RAG), fine-tuning, or AI agent development
• Experience using managed ML/AI solutions such as AWS SageMaker AI or Amazon Bedrock
• Demonstrated track record of scientific innovation with measurable business impact
• Experience building production systems that operate at scale
• Proven ability to drive scientific roadmap and secure management buy-in for new initiatives
• Strong collaboration skills with ability to influence across organizational boundaries

PREFERRED QUALIFICATIONS

• Experience with modeling tools such as R, scikit-learn, Spark MLLib, MxNet, Tensorflow, numpy, scipy etc.
• Experience with large scale distributed systems such as Hadoop, Spark etc.
• 7+ years of experience applying scientific methods to solve complex AI problems at scale
• Experience mentoring junior scientists and engineers
• Advanced knowledge of large language models, including fine-tuning, evaluation, and responsible AI practices
• Experience developing bias detection, fairness metrics, and ethical evaluation frameworks for AI systems
• Proven track record building reinforcement learning systems with human feedback, including annotation frameworks and reward modeling
• Experience creating evaluation frameworks measuring factuality, robustness, and safety across diverse scenarios, comparable to HELM or HealthBench
• Expertise in building reusable AI components with well-defined interfaces

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $150,400/year in our lowest geographic market up to $260,000/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.