Engineering Manager, AI

Jobtailor; New York (NY); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Hybrid schedule

In-office 3 days/week (MWF) in NYC

Job summary

Jobtailor in New York seeks an experienced engineering leader to lead and grow an AI-enabled infrastructure team. You will own technical architecture for workflow agents, drive reliability, and partner with product to set release gates.

You will build evaluation infrastructure, scale internal tooling, and hire engineers, reporting to the VP of Engineering. This hybrid NYC role emphasizes delivery, measurable improvements, and strong leadership across cross-functional teams.

Qualifications

• 5+ years in software engineering, including 1+ year in engineering leadership.
• Direct experience building or shipping a production AI/agentic system and owning its infrastructure.
• Hands-on familiarity with evaluation infrastructure: golden data sets, LLM-as-judge, human annotation, or online monitoring.
• Strong system design and architecture fundamentals.
• Ability to lead through ambiguity and turn unclear, fast-moving problems into structured execution.
• Deep technical fluency in code review, architecture review, and model/system tradeoffs.
• Strong communication skills across technical and non-technical stakeholders, especially product.
• High standards for engineering quality, reliability, and performance.
• Bonus: experience in workflow-heavy, enterprise SaaS, or infrastructure products.
• Bonus: exposure to complex regulated environments.
• Willingness to work a hybrid schedule with 3 in-office days per week (MWF) in New York City
• Authorization to work in the United States or ability to address U.S. sponsorship requirements

Responsibilities

• Lead and grow an engineering team building and hardening AI infrastructure.
• Build engineering culture through clear expectations, 1:1s, feedback loops, and career development.
• Own technical architecture for workflow agents, including orchestration, model serving, latency, cost per task, and failure handling.
• Partner with the Product Manager, AI to define reliability standards and release gates.
• Build and scale evaluation infrastructure, including golden data sets, LLM-as-judge pipelines, human annotation tooling, and online monitoring.
• Drive internal tooling for testing, error analysis, and hill-climbing agentic work.
• Improve reliability, observability, and engineering efficiency across the AI stack.
• Hire and develop engineers with ML/AI systems depth and future team leaders.
• Own the roadmap from technical direction and execution planning through delivery and measurable production accuracy and reliability gains.
• Report directly to the VP of Engineering and partner closely with the Product Manager, AI

Skills

Engineering Leadership

AI/Agentic System Infrastructure

Evaluation Infrastructure

System Design

Architecture Review

Reliability Standards

Performance Optimization

Team Development

Tools

Golden Data Sets

LLM-as-Judge Pipelines

Human Annotation Tooling

Online Monitoring

Internal Tooling

Model Serving

Job description

• Lead and grow an engineering team building and hardening PermitFlow's agentic infrastructure


• Build engineering culture through clear expectations, 1:1s, feedback loops, and career development


• Own technical architecture for workflow agents, including orchestration, model serving, latency, cost per task, and failure handling


• Partner with the Product Manager, AI to define reliability standards and release gates


• Build and scale evaluation infrastructure, including golden data sets, LLM-as-judge pipelines, human annotation tooling, and online monitoring


• Drive internal tooling for testing, error analysis, and hill-climbing agentic work


• Improve reliability, observability, and engineering efficiency across the AI stack


• Hire and develop engineers with ML/AI systems depth and future team leaders


• Own the roadmap from technical direction and execution planning through delivery and measurable production accuracy and reliability gains


• Report directly to the VP of Engineering and partner closely with the Product Manager, AI

Requirements

• 5+ years in software engineering, including 1+ year in engineering leadership


• Direct experience building or shipping a production AI/agentic system and owning its infrastructure


• Hands-on familiarity with evaluation infrastructure: golden data sets, LLM-as-judge, human annotation, or online monitoring


• Strong system design and architecture fundamentals


• Ability to lead through ambiguity and turn unclear, fast-moving problems into structured execution


• Deep technical fluency in code review, architecture review, and model/system tradeoffs


• Strong communication skills across technical and non-technical stakeholders, especially product


• High standards for engineering quality, reliability, and performance


• Bonus: experience in workflow-heavy, enterprise SaaS, or infrastructure products


• Bonus: exposure to complex regulated environments


• Willingness to work a hybrid schedule with 3 in-office days per week (MWF) in New York City


• Authorization to work in the United States or ability to address U.S. sponsorship requirements

Core Competencies

Demonstrates expertise in leading engineering teams and building AI/agentic systems, with a strong focus on technical architecture, reliability standards, and engineering efficiency. Proven ability to develop talent and drive high-quality engineering practices in a fast-paced environment.

Highest-signal resume keywords

• Engineering Leadership


• AI/Agentic System Infrastructure


• Evaluation Infrastructure Development


• System Design and Architecture


• Strong Communication Skills

Hard Skills

• Software Engineering


• Technical Architecture


• Model Serving


• Error Analysis


• Code Review


• Architecture Review


• Reliability Standards


• Performance Optimization


• Production Accuracy


• Latency Management

Soft Skills

• Team Development


• Feedback Loops


• Structured Execution


• Collaboration


• Problem-Solving

Industry Keywords

• Enterprise SaaS


• Workflow-Heavy Products


• Regulated Environments

Tools & Technologies

• Golden Data Sets


• LLM-as-Judge Pipelines


• Human Annotation Tooling


• Online Monitoring


• Internal Tooling