Principal Machine Learning Engineer

Harnham; San Francisco (CA); On-site; USD 450,000 - 550,000; Full time

Job summary

Harnham in the Bay Area is seeking a Principal Machine Learning Engineer to design and scale LLM-powered systems and reusable APIs. The role sits at the intersection of research and production, delivering architectures that support chatbots, code generation, and data interaction at enterprise scale.

You will bridge research with production-grade engineering, build agent-based systems and RAG pipelines, and work closely with cloud/SRE teams to ensure robust, scalable solutions.

Qualifications

• PhD in CS/Math/Stats or related field.
• Strong experience as an IC in ML engineering.
• Proven track record in building ML/AI systems in production.
• Deep understanding of ML fundamentals (NLP/CV).
• Experience with distributed systems (Ray, Horovod, DeepSpeed, etc.).
• Strong software engineering and system design fundamentals.
• Experience owning ML services end-to-end in enterprise environments.
• Ability to align technical work with business outcomes and communicate with stakeholders.

Responsibilities

• Design and deploy scalable LLM-powered systems and reusable backend APIs
• Bridge cutting-edge AI research with production-grade engineering
• Build agent-based systems, RAG pipelines, and workflow automation tools
• Develop capabilities across chatbots, text-to-code, and data interaction layers
• Partner with cloud and SRE teams to deliver robust, scalable architectures
• Implement distributed systems for training and inference (e.g., Ray, DeepSpeed)
• Drive best practices for ML systems, deployment, and performance optimization

Skills

IC ML Engineer

Distributed systems

ML deployment

NLP/CV fundamentals

Education

PhD (CS/Math/Stats)

Tools

Ray

Horovod

DeepSpeed

Kubeflow

Job description

Title: Principal Machine Learning Engineer

Location: Bay Area (5-days on-site)

Compensation: up to $500k + Bonus + Equity

We’re partnering with a globally recognized enterprise investing heavily in AI, building one of the most widely adopted internal LLM platforms at scale. The platform is used by a large global workforce and supports use cases such as chatbots, agent workflows, code generation, and data interaction, helping teams move from idea to production significantly faster.

This is a high-impact, highly visible role at the intersection of research and production, where you will help design and scale LLM-powered systems and reusable APIs used across the organization. The team operates in a fast-paced, execution-focused environment, taking problems from concept to production quickly with real adoption at scale.

What You’ll Do

• Design and deploy scalable LLM-powered systems and reusable backend APIs
• Bridge cutting-edge AI research with production-grade engineering
• Build agent-based systems, RAG pipelines, and workflow automation tools
• Develop capabilities across chatbots, text-to-code, and data interaction layers
• Partner with cloud and SRE teams to deliver robust, scalable architectures
• Implement distributed systems for training and inference (e.g., Ray, DeepSpeed)
• Drive best practices for ML systems, deployment, and performance optimization

Requirements

• PhD in Computer Science, Mathematics, Statistics, or related field
• Strong experience as an IC in Machine Learning Engineering
• Proven track record in building and scaling ML/AI systems in production
• Deep understanding of ML fundamentals (NLP and/or Computer Vision)
• Experience with distributed systems (Ray, Horovod, DeepSpeed, etc.)
• Strong software engineering and system design fundamentals
• Experience owning ML services end-to-end in enterprise environments
• Ability to align technical work with business outcomes and communicate with stakeholders

Nice to Have

• Experience with ML pipelines (Kubeflow, DVC, Ray)
• Familiarity with microservices (gRPC, GraphQL)
• Experience with LLM optimization (fine-tuning, quantization, PEFT)
• Knowledge of advanced prompting strategies (Chain-of-Thought, etc.)

If you're interested in working on AI systems at a massive scale, with real adoption and immediate impact, this is a rare opportunity to do so within a highly respected, well-resourced environment.