Staff Software Engineer, AI Agents & Low-Latency Infra

Fdetimes; New York (NY); On-site; USD 231,000 - 340,000; Full time

Job summary

Harvey is building a groundbreaking AI-driven platform for legal professionals. We seek a Software Engineer, Agents, with extensive experience to design environments for agentic work, select models, manage context, and develop evaluative tools.

You’ll influence architectural choices and shipping high-impact features that redefine knowledge work in a fast-growing company. This role calls for deep expertise in Python, LLM integrations, and agent frameworks, plus a track record of shipping

Qualifications

• 10+ years (post-BS/MS) of software engineering experience.
• Passion for building effective domain-specific agents.
• Iterative mindset: you develop proof of concepts, make decisions quickly, and ship v0s.
• Comfortable with when and how to use evaluations to drive quality.
• Humble and adaptable about code and frameworks. We expect you to drive adoption of new best practices as they develop.
• We are hiring for this role across Mid, Senior, and Staff levels.
• Proficiency in Python and experience working with LLM APIs and agent frameworks.
• Experience with shipping user-facing products, either on the backend or full-stack.

Responsibilities

• Partner with customers and PMs to understand legal workflows, design practical evaluations that capture what "excellent" means, and ship agents that get the job done.
• Optimize agent performance through prompt engineering, model selection, tool design, skill writing, context window management, and eval harness development.
• Work with our model infra team to design and implement infrastructure for low-latency agent execution, including caching strategies, parallel tool calls, or subagent patterns.
• Improve our observability and instrumentation to profile agent behavior, identify bottlenecks, and drive optimization decisions.
• Stay current on new developments in agentic systems and bring those learnings back to the products we build.

Skills

Python

LLM APIs

Agent frameworks

Job description

Harvey is building a groundbreaking AI-driven platform for legal professionals. We seek a Software Engineer, Agents, with extensive experience to design environments for agentic work, select models, manage context, and develop evaluative tools.

You’ll influence architectural choices and shipping high-impact features that redefine knowledge work in a fast-growing company. This role calls for deep expertise in Python, LLM integrations, and agent frameworks, plus a track record of shipping