Principal AI Engineer — Build Production-Grade AI Systems

Socket.dev; San Francisco (CA); On-site; USD 260,000 - 340,000; Full time

Benefits offered by this job

Generous PTO

Parental Leave

Rewards & Recognition

Insurance Benefits

Job summary

Innovaccer seeks a Principal AI Engineer to build production-grade AI systems at scale. You will take ideas from paper to prototype to production, design multi-model pipelines, and ensure product-level accuracy through rigorous evaluation.

Role requires deep expertise in Python, PyTorch, and distributed training, with prior production ML experience. You will mentor engineers and work with clinicians and operators to ensure reliable outputs.

Qualifications

• MS or PhD in Computer Science, Machine Learning, or related quantitative field; exceptional BS candidates considered.
• First-author publications at NeurIPS/ICML/ICLR/ACL/EMNLP or similar; open-source ML contributions; model training and shipping experience.
• Fine-tuned open-weight models; explain choice between parameter-efficient vs full fine-tuning.
• Strong Python and PyTorch; familiarity with training and serving stacks (HuggingFace, DeepSpeed, vLLM or equivalents).
• Exposure to multi-GPU training and understanding sharding strategy and its importance.
• Experience finishing projects; production-ready models; data pipelines for training data (dedup, filtering, decontamination).
• Ability to scope ambiguous problems into plans; mentoring or direction for other engineers is a plus.

Responsibilities

• Take ideas from paper to prototype to production and build real product model layers.
• Choose model sizes, compose models into a working system, and maintain a product-level accuracy bar.
• Write real code in Python and PyTorch; optimize training runs and system efficiency.
• Design experiments with hypotheses, ablations, and report results.
• Explain work to clinicians and operators who will rely on outputs.
• Lead post-training on large models and own outcomes across experiments.

Skills

Python

PyTorch

Distributed training

Model deployment

Research publications

Multi-GPU training

System design

Mentoring

Education

MS or PhD in Computer Science / ML

Tools

HuggingFace

DeepSpeed

FSDP

vLLM

SGLang

Job description

Innovaccer seeks a Principal AI Engineer to build production-grade AI systems at scale. You will take ideas from paper to prototype to production, design multi-model pipelines, and ensure product-level accuracy through rigorous evaluation.

Role requires deep expertise in Python, PyTorch, and distributed training, with prior production ML experience. You will mentor engineers and work with clinicians and operators to ensure reliable outputs.