Agentic AI Scientist - LLMs, Tool Use & Orchestration

Socket.dev; Bellevue (WA); On-site; USD 143,000 - 193,000; Full time

Job summary

Amazon Fulfillment Technologies in Bellevue, WA seeks an Applied Scientist to lead the development of agentic systems that assist with operational decision making and orchestration. You will build full agentic systems using multi-agent orchestration, tool use, memory, and action execution.

Training of LLMs will involve rejection sampling, SFT, continual post-training, and RL. Responsibilities include data generation for reasoning trajectories, reward modeling, policy optimization for LLMs,

Qualifications

• PhD or Master’s degree with 4+ years of CS/CE/ML related experience.
• Experience building models for business applications.
• Proficiency programming in Java, C++, Python.

Responsibilities

• Lead development of agentic systems for operational decision making and orchestration.
• Generate training and preference data (reasoning trajectories, tool traces).
• Perform reward modeling and policy optimization for LLMs (DPO, IPO, RLHF/RLAIF with PPO/GRPO).
• Fine-tune supervised trajectories and tool-use traces.
• Advance Continual Learning and Verbal Reinforcement Learning.
• Evaluate decision quality metrics and scale LLM-based evaluations.

Skills

Java

C++

Python

Algorithms

Data mining

Education

PhD or MS with 4+ years in CS/CE/ML

Tools

Unix/Linux

Job description

Amazon Fulfillment Technologies in Bellevue, WA seeks an Applied Scientist to lead the development of agentic systems that assist with operational decision making and orchestration. You will build full agentic systems using multi-agent orchestration, tool use, memory, and action execution.

Training of LLMs will involve rejection sampling, SFT, continual post-training, and RL. Responsibilities include data generation for reasoning trajectories, reward modeling, policy optimization for LLMs,