Staff AI Engineer — Production LLM Agents & AI Infra

People in AI; New York (NY); On-site; USD 250,000 - 275,000; Full time

Job summary

People in AI in New York City is seeking a hands-on engineer to design, build, and deploy production-grade LLM-powered agents at scale. You will shape how intelligent agents are designed, evaluated, deployed, and continuously improved in a data-rich production environment.

The role focuses on reliable AI systems that deliver measurable outcomes, collaborating with engineers and product teams to drive architecture and tooling decisions across the platform.

Qualifications

• Strong software engineering background with hands-on Python experience.
• Experience building and deploying LLM-powered agents in production.
• Experience with evaluation frameworks, automated testing, or agent reliability systems.
• Understanding of agent orchestration, tool use, context management, and failure handling.

Responsibilities

• Design, develop, and deploy production-grade LLM agents with tool calling and guardrails.
• Build evaluation infrastructure, including golden datasets and automated tests.
• Develop observability to monitor agent performance, latency, and cost.
• Identify and resolve production failure modes to improve reliability.
• Benchmark LLMs and architectures to inform decisions.
• Collaborate with engineers and product stakeholders to scale AI-powered workflows.
• Shape architecture and establish best practices for reliable agent systems.

Skills

Python

Production backend

LLM-powered agents

Agent orchestration

Observability

Performance benchmarking

Tools

Testing frameworks

Job description

People in AI in New York City is seeking a hands-on engineer to design, build, and deploy production-grade LLM-powered agents at scale. You will shape how intelligent agents are designed, evaluated, deployed, and continuously improved in a data-rich production environment.

The role focuses on reliable AI systems that deliver measurable outcomes, collaborating with engineers and product teams to drive architecture and tooling decisions across the platform.