Software Engineer, AI Agent Optimization

Remote Genie; United States; Remote; USD 110,000 - 325,000; Full time

Benefits offered by this job

Healthcare benefits

401k with employer match

Global benefits

Family planning support

Mental health & coaching

Job summary

Reddit, Inc. is seeking a Software Engineer to lead its agent optimization and evaluation platform. You will build systems to measure agents’ impact on engineering velocity, and collaborate with Platform Engineering and Infrastructure to integrate AI capabilities into workflows.

You will design evaluation environments, implement offline replay, and define metrics to improve quality, latency, and cost across complex distributed systems.

Qualifications

• 5-12+ years of engineering experience.
• Extensive experience building and shipping large-scale production software systems.
• Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI, search, developer platforms, or other complex distributed systems.
• Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them effectively.
• Experience building systems that optimize across multiple dimensions, including quality, latency, reliability, and cost.
• Excellent software engineering fundamentals and a passion for building production‑quality systems.
• Ability to operate independently in ambiguous environments and drive technical direction for highly strategic initiatives.
• Strong cross‑functional collaboration skills and experience working across engineering, infrastructure, and platform organizations.

Responsibilities

• Design and build Reddit’s agent optimization harness for evaluating, benchmarking, and improving AI agents across real software engineering workflows.
• Develop Reddit-specific evaluation environments that accurately measure agent performance against meaningful engineering tasks.
• Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback.
• Design model routing and orchestration strategies that optimize for quality, latency, and cost.
• Define metrics and experimentation frameworks that measure developer productivity and engineering velocity.
• Build tooling to analyze agent behavior, investigate failures, and accelerate iteration.
• Partner closely with Platform Engineering, Infrastructure, and Developer Productivity teams to integrate agent capabilities into engineering workflows.
• Help shape Reddit’s long‑term strategy for a flexible, model‑agnostic agent platform that gives us control over our AI capabilities and economics

Skills

5-12+ years of engineering experience

Large-scale production software

AI/evaluation systems design

LLMs & AI agents

Multi-dimensional optimization

Production-grade software

Independence in ambiguity

Cross-functional collaboration

Job description

Reddit, Inc. is seeking a Software Engineer to lead its agent optimization and evaluation platform. You will build systems to measure agents’ impact on engineering velocity, and collaborate with Platform Engineering and Infrastructure to integrate AI capabilities into workflows.

You will design evaluation environments, implement offline replay, and define metrics to improve quality, latency, and cost across complex distributed systems.