Staff Software Engineer

People in AI; New York (NY); On-site; USD 180,000 - 240,000; Full time

Job summary

People in AI is seeking a Staff Software Engineer to lead production AI agents and model adaptation systems in a fast-growing startup environment. You will own LLM strategy, fine-tune models, and architect reliable inference infrastructure while mentoring engineers.

You will work hands-on with Python/PyTorch, explore open-weight models, and guide evaluation pipelines to reduce hallucinations and improve performance in real-world use cases.

Qualifications

• Strong background in applied AI, ML, or software engineering.
• Proven experience fine-tuning LLMs and deploying adapted models in production.
• Experience building production-grade AI agents and evaluation pipelines.
• Excellent Python and PyTorch skills.

Responsibilities

• Own model strategy across LLMs, balancing accuracy, latency, and cost.
• Fine-tune and adapt open-weight models using LoRA, QLoRA, SFT, and preference optimization.
• Design and deploy production AI agents with tool use and orchestration.
• Build evaluation frameworks to measure performance, reliability, and hallucinations.
• Turn real-world feedback into training data and continuous improvements.
• Optimize model serving, inference performance, and quantization.
• Drive technical architecture and mentor engineers.

Skills

Python

PyTorch

LLM fine-tuning

Production systems

Open-weight models

Tools

Hugging Face

DeepSpeed

Ray

LLM evaluation

Job description

We're partnering with a fast-growing, venture-backed AI startup building intelligent systems to transform complex, data-intensive investigations.

With strong commercial traction, growing enterprise adoption, and significant technical challenges ahead, the company is looking for a Staff Software Engineer to lead the development of production AI agents and model adaptation systems.

This is a hands-on technical leadership role with ownership across LLM fine-tuning, agent architecture, evaluation, and inference infrastructure.

What You'll Do

• Own model strategy, balancing frontier and open-weight LLMs across accuracy, latency, and cost.
• Fine-tune and adapt open-weight models using LoRA, QLoRA, SFT, and preference optimization.
• Design and deploy production AI agents with tool use, orchestration, and structured outputs.
• Build evaluation frameworks to measure model performance, reliability, and hallucinations.
• Turn real-world user feedback and production traces into training data and continuous model improvements.
• Optimize model serving, inference performance, batching, and quantization.
• Drive technical architecture, mentor engineers, and remain hands-on in code.

What You'll Bring

• Strong background in applied AI, machine learning, or software engineering.
• Proven experience fine-tuning LLMs and deploying adapted models in production.
• Experience building production-grade AI agents and evaluation pipelines.
• Strong Python and PyTorch skills.
• Familiarity with Hugging Face, DeepSpeed, Ray, or similar frameworks.
• Understanding of LLM inference, model serving, observability, and cost optimization.
• Ability to lead technical decisions and solve ambiguous engineering challenges.
• Python, PyTorch
• LoRA, QLoRA, SFT, preference optimization
• LLM evaluation, observability, and guardrails

Why Join?

• Own the technical direction of AI agents and model adaptation at a growing startup.
• Work on challenging, real-world AI problems with proprietary data and direct customer feedback.
• Build and deploy sophisticated AI systems beyond proof-of-concept.
• Join a venture-backed, high-performing engineering team with strong commercial momentum.
• Receive meaningful equity and have a direct impact on product and company growth.