Principal, Machine Learning Engineer

aijoblist; San Francisco (CA); On-site; USD 252,000 - 374,000; Full time

Benefits offered by this job

Competitive base compensation

Generous early-stage equity

Comprehensive benefits program

Flexible time off

Paid parental leave

Job summary

Lila Sciences in San Francisco is looking for a Principal ML Engineer to design and scale the machine learning infrastructure for scientific applications. This role involves developing large-scale training pipelines, managing end-to-end ML systems, and collaborating across teams to bridge the gap between machine learning research and production. Candidates should have a Master's in Computer Science or a related field, 10+ years in production ML, and deep expertise in distributed training systems. A competitive compensation package is offered.

Qualifications

• 10+ years of hands-on experience building and operating production ML systems at scale.
• Deep expertise in distributed training infrastructure with large-scale GPU clusters.
• Strong fundamentals in system design, production-grade code, and CI/CD.

Responsibilities

• Design, build, and optimize large-scale training pipelines for generative models.
• Own production ML systems from deployment to monitoring.
• Collaborate with AI scientists to bridge research and deployment.

Skills

Production ML systems

Distributed training infrastructure

ML frameworks (PyTorch, JAX, TensorFlow)

Software engineering fundamentals

Cross-functional collaboration

Education

Master's degree in Computer Science or related field

Job description

Lila is building a platform where AI and automation co-evolve to solve the hardest problems in medicine. Within Life Science AI (LSAI), ML engineers build and operate the systems that turn generative models and reasoning frameworks into production capabilities powering automated scientific discovery across Lila's life science domains.

We are seeking a Principal ML Engineer to design, build, and scale the ML infrastructure behind models spanning biological sequence design, molecular structure prediction, antibody engineering, and multimodal scientific reasoning. You will own critical systems end to end, from training pipelines and distributed compute to model deployment and integration into Lila's closed-loop discovery engine.

This is a high-impact IC role for someone who operates at the intersection of ML systems engineering and life science applications. You will shape the technical direction for how ML models are trained, evaluated, and deployed at scale, collaborate closely with AI scientists and experimental researchers to close the computational-experimental loop, and drive Lila's ML infrastructure toward the next generation of capabilities.

What You'll Be Building

• Design, build, and optimize large-scale training pipelines for generative models on biological and chemical data, including distributed training across GPU clusters
• Own production ML systems end to end: model deployment, serving infrastructure, monitoring, and reliability for models used in Lila's scientific workflows
• Architect ML infrastructure that supports rapid iteration across sequence design, structure prediction, and multimodal scientific reasoning workloads
• Drive the engineering side of Lila's "Lab-in-the-Loop" lifecycle: build pipeline models, integrate experimental feedback loops, and ensure model outputs are actionable for downstream scientific workflows
• Define and advance ML engineering standards, tooling, and best practices across the AI organization
• Collaborate with AI scientists to translate research prototypes into robust, scalable production systems, bridging the research-to-deployment gap

What You’ll Need to Succeed

• Master's degree or higher in Computer Science, Machine Learning, or a related quantitative field (or Bachelor's with equivalent professional experience)
• 10+ years of hands‑on experience building and operating production ML systems at scale
• Deep expertise in distributed training infrastructure, including experience with large-scale GPU clusters (AWS, GCP, or on‑prem)
• Strong software engineering fundamentals: system design, production‑grade code, CI/CD, observability, and reliability practices
• Proficiency in ML frameworks (PyTorch, JAX, or TensorFlow) with experience optimizing training and inference performance
• Demonstrated ability to drive technical direction for ML infrastructure independently, from architecture through implementation
• Track record of cross‑functional collaboration with research scientists, translating between ML methodology and engineering execution

Bonus Points For

• Experience building training or inference infrastructure for generative models applied to biological sequences, molecular structures, or scientific data
• Experience with agentic frameworks, active learning loops, or closed‑loop experimental workflows
• Contributions to open-source ML tools, frameworks, or infrastructure projects
• Familiarity with at least one life science domain (molecular biology, genomics, protein engineering, or nucleic acid design)
• Experience with model evaluation frameworks for scientific applications where ground truth is sparse or delayed

Compensation

We offer competitive base compensation with bonus potential and generous early‑stage equity. Your final offer will reflect your background, expertise, and expected impact.

U.S. Benefits.

Full‑time U.S. employees receive a comprehensive benefits program including medical, dental, and vision coverage; employer‑paid life and disability insurance; flexible time off with generous company wide holidays; paid parental leave; an educational assistance program; commuter benefits, including bike share memberships for office based employees; and a company subsidized lunch program.

International Benefits.

Full‑time employees outside the U.S. receive a comprehensive benefits program tailored to their region. USD salary ranges apply only to U.S.-based positions; international salaries are set to local market.

Expected Base Salary Range

$252,000 - $374,000 USD

Equal Employment Opportunity

Lila Sciences is committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status.