LLM Agent Evaluation Engineer Intern (MaaS)

ByteDance; Seattle (WA); On-site; USD 58,000 - 61,000; Full time

Benefits offered by this job

Health insurance

10 paid holidays per year

Housing allowance

Job summary

ByteDance is offering an internship on our Applied Machine Learning Ark team. You will work on end-to-end LLM service platforms, from evaluation to production integration, focusing on MaaS for both providers and downstream users.

You will design evaluation systems for LLM-based agents, build benchmarks, and analyze logs to drive improvements, gaining hands-on experience in large-scale AI systems and data-driven feedback loops.

Qualifications

• Currently pursuing a Bachelor's or Master's in CS/AI/ML/Data Science or related field.
• Solid foundation in machine learning and deep learning principles.
• Hands-on experience with LLM-based systems via research, internships or projects.
• Strong Python skills and familiarity with ML/agent evaluation frameworks.
• Demonstrated research or engineering ability through publications or projects.

Responsibilities

• Design evaluation systems for LLM-based agents covering success, tool use and reliability.
• Build benchmarks and automated judging pipelines with rule-based and model-based checks.
• Analyze agent traces and user feedback to identify failure patterns for improvements.
• Collaborate with research and product teams to deploy methods into production.

Skills

ML fundamentals

Python programming

LLM systems experience

Research capabilities

Education

Bachelor's/Master's in CS/AI/ML/Data Science

Tools

ML frameworks

Job description

ByteDance is offering an internship on our Applied Machine Learning Ark team. You will work on end-to-end LLM service platforms, from evaluation to production integration, focusing on MaaS for both providers and downstream users.

You will design evaluation systems for LLM-based agents, build benchmarks, and analyze logs to drive improvements, gaining hands-on experience in large-scale AI systems and data-driven feedback loops.