LLM Agent Evaluation & Evolution Researcher

Bytedance; San Jose (CA); Hybrid; USD 180,000 - 240,000; Full time

Job summary

ByteDance's Applied Machine Learning Ark team in San Jose is seeking researchers and engineers specializing in Large Language Models and AI agents. You will work on end-to-end MaaS platforms, spanning model serving, tool usage, and evaluation pipelines.

Successful candidates will contribute to design, development, and operation across US and international markets, with emphasis on production-ready research and data-driven improvements.

Qualifications

• PhD in CS/AI/ML/Data Science or related field, completed or near completion.
• Strong foundation in ML and deep learning with practical ML exposure.
• Experience with LLM-based systems, agents, tool use and retrieval.
• Proven research track record through publications or projects.

Responsibilities

• Design evaluation systems for LLM-based agents, covering task success, tool use, reasoning quality, and reliability.
• Build benchmarks and automated judging pipelines, combining rule-based checks, model-based judging, and human review, etc.
• Analyze agent execution traces and user feedback to identify failure patterns and turn them into concrete system improvements.
• Support the closed loop from experience to capability, and work with research, platform, and product teams to bring methods into production.

Skills

ML fundamentals

DL fundamentals

LLM-based systems

Python programming

Research publications

Evaluation frameworks

Multi-agent systems

Education

PhD in Computer Science / AI / ML / Data Science

Job description

ByteDance's Applied Machine Learning Ark team in San Jose is seeking researchers and engineers specializing in Large Language Models and AI agents. You will work on end-to-end MaaS platforms, spanning model serving, tool usage, and evaluation pipelines.

Successful candidates will contribute to design, development, and operation across US and international markets, with emphasis on production-ready research and data-driven improvements.