Principal AI Engineer: Lead Production-Ready AI at Scale

Innovaccer; San Francisco (CA); On-site; USD 210,000 - 300,000; Full time

Benefits offered by this job

PTO 20 days/year

Parental leave

Rewards & Recognition

Insurance benefits and disability

Job summary

Innovaccer is seeking a Principal AI Engineer to build production-grade AI systems, including LLM-based solutions, AI agents, and RAG-driven workflows. You will collaborate with product, engineering, and data teams to design, develop, and deploy AI-powered applications at scale.

The ideal candidate will own the full AI development lifecycle from experimentation to deployment, mentor others, and set technical direction for in-house modeling efforts.

Qualifications

• MS or PhD in Computer Science, Machine Learning, or a related quantitative field.
• Depth beyond coursework: first-author publications or meaningful open-source ML contributions.
• Experience fine-tuning open-weight models and understanding parameter-efficient vs full fine-tuning.
• Strong Python and PyTorch; familiarity with serving stacks and multi-GPU training.
• Experience with data pipelines feeding large training runs.

Responsibilities

• Take ideas from paper to prototype to production.
• Build the model layer of a real product and ensure production-ready accuracy.
• Write real code in Python and PyTorch; optimize training runs.
• Design and run experiments; state hypotheses and report results.
• Measure results and validate evals before deployment.
• Read current research and judge applicability of techniques.
• Explain work to non-AI engineers including clinicians and operators.

Skills

Python

PyTorch

Distributed training

Experiment design

Communicating with non-AI stakeholders

Mentoring

Education

MS or PhD in Computer Science / ML

Tools

HuggingFace

DeepSpeed

vLLM

SGLang

FSDP

Job description

Innovaccer is seeking a Principal AI Engineer to build production-grade AI systems, including LLM-based solutions, AI agents, and RAG-driven workflows. You will collaborate with product, engineering, and data teams to design, develop, and deploy AI-powered applications at scale.

The ideal candidate will own the full AI development lifecycle from experimentation to deployment, mentor others, and set technical direction for in-house modeling efforts.