Research Engineer, AI Agent Engine — Production Benchmarks

LangChain, Inc.; New York (NY); On-site; USD 150,000 - 230,000; Full time

Benefits offered by this job

Competitive compensation

Equity

Medical, dental, vision coverage

401(k) plan

Job summary

LangChain, Inc. is seeking an experienced research engineer to advance the LangSmith Engine, building proactive agents and improving production performance.

You’ll study real agent failures, run benchmarks, and turn ideas into production improvements with a focus on cost, latency, reliability, and scalability. Join a team of engineers and researchers in a role that blends research autonomy with practical impact, from model selection and fine-tuning to post-training techniques and

Qualifications

• 4+ years of ML/AI research or closely related field.
• Master’s or Ph.D. in a relevant field.
• Hands-on experience with LLMs and AI agents, including analyzing model behavior and improving performance.
• Strong experience designing benchmarks, evaluations, and experiments for AI/ML systems.

Responsibilities

• Build and maintain benchmarks and evaluations for LangSmith Engine on real-world tasks.
• Design and run experiments to improve agent performance across models, prompting, context, tools, orchestration, and strategies.
• Explore and implement post-training and fine-tuning techniques when they meaningfully improve capabilities, quality, or cost.
• Turn successful experiments into production improvements in collaboration with engineers and researchers.
• Help define the ML roadmap and provide technical leadership to the team.

Skills

4+ years ML/AI research

LLMs & AI agents

Benchmarks & experiments

Software engineering

Technical leadership

Education

Master's or Ph.D. in a relevant field

Tools

LLM platforms

GPU infrastructure

Job description

LangChain, Inc. is seeking an experienced research engineer to advance the LangSmith Engine, building proactive agents and improving production performance.

You’ll study real agent failures, run benchmarks, and turn ideas into production improvements with a focus on cost, latency, reliability, and scalability. Join a team of engineers and researchers in a role that blends research autonomy with practical impact, from model selection and fine-tuning to post-training techniques and