Staff AI/ML Engineer, Agents & Model Adaptation

People in AI; New York (NY); On-site; USD 225,000 - 275,000; Full time

Job summary

People in AI in New York City seeks a Staff Applied AI/ML Engineer to shape agent architecture and model strategy while remaining hands-on. You’ll decide when frontier models or adapted models are best and prove approaches in production.

You’ll design agent systems, tune models, build evaluation harnesses, and mentor engineers, delivering reliable, scalable AI that handles messy data and real-world consequences.

Qualifications

• Experience shipping production AI systems.
• Hands-on LLM fine-tuning with clear training choices and results.
• Ability to own technical direction and mentor engineers.

Responsibilities

• Design and build agent systems spanning tool use, context management, orchestration, and guardrails.
• Choose models based on accuracy, latency, cost, and operational requirements.
• Fine-tune open-weight models using LoRA, QLoRA, supervised fine-tuning, and preference optimization.
• Turn user corrections and traces into training data and evaluation cases.
• Build evaluation harnesses measuring accuracy, hallucinations, false positives, and task completion.
• Own production serving decisions including batching, quantization, monitoring, and inference efficiency.
• Test systems against prompt injection, data leakage, and adversarial inputs.
• Guide architecture, review designs, and mentor engineers while continuing to write code.

Skills

Software engineering

Agent systems

LLM fine-tuning

Evaluation design

Architecture direction

Mentoring engineers

Reliability/security/cost

Tools

Python

PyTorch

Job description

Staff Applied AI/ML Engineer, Agents & Model Adaptation

Compensation: $225K-$275K base + equity

Location: New York City | Five days onsite

A venture-backed AI company helping businesses investigate complex information and automate evidence-based workflows.

The company is building AI systems that connect fragmented records, identify relationships, and help people make better decisions. The engineering challenge is making agents accurate, reliable, and useful when the data is messy and mistakes carry real consequences.

As a Staff Applied AI Engineer, you’ll shape the agent architecture and model strategy while staying hands-on. You’ll help decide when frontier models are the right choice, when smaller adapted models make more sense, and how to prove that either approach works in production.

What You’ll Do

• Design and build agent systems spanning tool use, context management, orchestration, and guardrails.
• Choose models based on accuracy, latency, cost, and operational requirements.
• Fine-tune open-weight models using approaches such as LoRA, QLoRA, supervised fine-tuning, and preference optimization.
• Turn user corrections and production traces into training data and evaluation cases.
• Build evaluation harnesses that measure accuracy, hallucinations, false positives, and task completion.
• Own production serving decisions, including batching, quantization, monitoring, and inference efficiency.
• Test systems against prompt injection, data leakage, and adversarial inputs.
• Guide architecture, review designs, and mentor engineers while continuing to write code.

What You’ll Bring

• Software engineering or applied ML experience, including technical ownership of production AI systems.
• Experience shipping agents that use external tools and produce structured outputs.
• Hands-on LLM fine-tuning experience, with clear examples of training choices, evaluation results, and production outcomes.
• Strong Python and PyTorch skills.
• Experience building evaluations that informed model selection, releases, or architectural decisions.
• Sound judgment across reliability, security, performance, and cost.
• The ability to set technical direction and bring other engineers along.

Experience with entity resolution, graph ML, model distillation, or domain-specific benchmarks would be useful.