Model Infrastructure Engineering Lead

Harvey; United States; On-site; USD 260,000 - 340,000; Full time

Job summary

Harvey is seeking an Engineering Manager for Model Infrastructure to lead a high-performing team responsible for the platform powering all model requests. You will collaborate with AI Research, Product Engineering, Infrastructure, and external providers to ensure reliability, scalability, and cost efficiency as the business expands.

The role offers the opportunity to shape Harvey's AI platform foundation, guide architectural decisions, and grow an organization through Harvey's next growth phase.

Qualifications

• 8+ years of software engineering experience with multiple years in engineering leadership.
• Experience leading teams for large-scale distributed systems or cloud infrastructure.
• Strong architectural guidance and mentorship for senior engineers.
• Experience running highly available production services with focus on reliability and operations.
• Experience building scalable platforms with observability, automation, and cost optimization.
• Strong cross-functional leadership across Engineering, Research, Product, and vendors.
• Excellent communication and ability to influence technical strategy across orgs.
• Passion for building teams and nurturing engineering talent.

Responsibilities

• Lead and grow a high-performing team of software engineers for Harvey's Model Infrastructure.
• Define the technical roadmap for reliability, scalability, and operational excellence.
• Build reliable systems for model provisioning, capacity management, failover, and incident response across multiple AI providers.
• Own Harvey's multi-provider model platform, including integrations, SDK upgrades, API migrations, and onboarding new providers.
• Drive evolution of the Unified Model Controller and Model Selector to route traffic based on health, latency, quality, compliance, and cost.
• Improve observability with dashboards, alerts, cost reporting, and end-to-end telemetry.
• Partner with Product Engineering on model launches, capacity planning, experimentation, and production monitoring.
• Lead initiatives to improve inference efficiency and reduce infrastructure costs.

Skills

Software engineering

Distributed systems

Cloud infrastructure

Leadership

Mentoring

Observability

Communication

Team building

Tools

Kubernetes

Spark

Kafka

Airflow

Job description

Harvey is seeking an Engineering Manager for Model Infrastructure to lead a high-performing team responsible for the platform powering all model requests. You will collaborate with AI Research, Product Engineering, Infrastructure, and external providers to ensure reliability, scalability, and cost efficiency as the business expands.

The role offers the opportunity to shape Harvey's AI platform foundation, guide architectural decisions, and grow an organization through Harvey's next growth phase.