Staff AI Engineer

People in AI; New York (NY); On-site; USD 250,000 - 275,000; Full time

Job summary

People in AI in New York City is seeking a hands-on engineer to design, build, and deploy production-grade LLM-powered agents at scale. You will shape how intelligent agents are designed, evaluated, deployed, and continuously improved in a data-rich production environment.

The role focuses on reliable AI systems that deliver measurable outcomes, collaborating with engineers and product teams to drive architecture and tooling decisions across the platform.

Qualifications

• Strong software engineering background with hands-on Python experience.
• Experience building and deploying LLM-powered agents in production.
• Experience with evaluation frameworks, automated testing, or agent reliability systems.
• Understanding of agent orchestration, tool use, context management, and failure handling.

Responsibilities

• Design, develop, and deploy production-grade LLM agents with tool calling and guardrails.
• Build evaluation infrastructure, including golden datasets and automated tests.
• Develop observability to monitor agent performance, latency, and cost.
• Identify and resolve production failure modes to improve reliability.
• Benchmark LLMs and architectures to inform decisions.
• Collaborate with engineers and product stakeholders to scale AI-powered workflows.
• Shape architecture and establish best practices for reliable agent systems.

Skills

Python

Production backend

LLM-powered agents

Agent orchestration

Observability

Performance benchmarking

Tools

Testing frameworks

Job description

Compensation: $250,000 - $275,000 base + equity

Location: New York City, Onsite

We're partnering with an ambitious, venture‑backed technology startup developing sophisticated AI systems to automate complex, data‑intensive workflows.

The company is building a next‑generation AI platform that combines large language models, autonomous agents, and advanced data infrastructure to solve challenging real‑world problems. With strong early commercial traction and backing from established investors, the business is expanding its core engineering team as it scales its technology and customer base.

The Opportunity

This is a hands‑on, high‑impact engineering role at the intersection of LLM agents, backend engineering, and AI infrastructure.

You'll help define how intelligent agents are designed, evaluated, deployed, and continuously improved in production. The focus is on building technically sophisticated, reliable AI systems that deliver measurable outcomes, rather than experimentation or proof‑of‑concept development.

What You'll Be Doing

• Design, develop, and deploy production‑grade LLM agents, including tool calling, orchestration, context management, and guardrails.
• Build evaluation infrastructure, including golden datasets, automated testing frameworks, LLM‑as‑judge systems, and regression testing.
• Develop observability and monitoring capabilities to track agent performance, accuracy, latency, and cost.
• Identify and resolve complex production failure modes to improve overall agent reliability.
• Benchmark LLMs, agent architectures, and emerging technologies to inform technical decisions.
• Collaborate closely with engineers and product stakeholders to develop scalable AI‑powered workflows.
• Influence technical architecture and establish best practices for building and deploying reliable agentic systems.

What We're Looking For

• Strong software engineering background with extensive hands‑on experience in Python and production backend development.
• Demonstrated experience building and deploying LLM‑powered agents in production environments.
• Practical experience developing evaluation frameworks, automated testing infrastructure, or agent reliability systems.
• Deep understanding of agent orchestration, tool use, context management, and failure handling.
• Familiarity with emerging agent technologies and protocols such as MCP, A2A, OpenHands, or similar frameworks.
• Experience with production observability, performance monitoring, and model benchmarking.
• Strong architectural judgment, technical ownership, and the ability to operate independently.
• Experience working in fast‑paced, high‑growth engineering environments.

Why Join?

• Technical ownership: Influence the architecture and development of sophisticated AI systems from an early stage.
• Production AI: Work on complex agentic systems solving meaningful real‑world problems.
• Early‑stage impact: Join a growing engineering organization where your technical decisions will directly shape the product.
• Strong momentum: Join a well‑funded business with meaningful early commercial traction.
• Collaborative engineering culture: Work alongside a highly technical team focused on building ambitious AI products.