Senior Pre-training Research Engineer - Foundation Models

Sciforium; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401k plan

Daily lunch, snacks, and beverages

Flexible time off

Competitive salary and equity

Job summary

Sciforium in San Francisco is seeking a Pre-training Research Engineer to advance byte-native and multimodal foundation models. You will implement training code, scale experiments, and contribute production-grade infrastructure to deliver high-quality base models for real-world use cases.

The role emphasizes rapid iteration, rigorous ablations, and collaboration with data and distributed training engineers to improve efficiency, reliability, and scalability across large GPU clusters.

Qualifications

• 5+ years of experience in ML research or engineering with a track record of pre-training foundation models.
• Strong software engineering skills to write robust training code.
• Solid understanding of deep learning fundamentals and pre-training methods.
• Ability to implement research ideas and evaluate with baselines and metrics.
• Hands-on experience with GPU-based training and distributed training workflows.

Responsibilities

• Train large byte-native and multimodal foundation models across massive corpora.
• Implement and evaluate new model architectures, training objectives, and optimizations.
• Develop stable pre-training recipes and run scaling experiments for novel architectures.
• Conduct ablations and analyze training dynamics and base-model quality.
• Work with data and distributed training engineers to improve training efficiency and scalability.

Skills

5+ years ML exp

Software engineering

ML Foundations

Research & experimentation

GPU & distributed training

Education

MS in CS/ML/AI/Math

Tools

JAX

Flax

XLA

FSDP

Megatron

Job description

Sciforium in San Francisco is seeking a Pre-training Research Engineer to advance byte-native and multimodal foundation models. You will implement training code, scale experiments, and contribute production-grade infrastructure to deliver high-quality base models for real-world use cases.

The role emphasizes rapid iteration, rigorous ablations, and collaboration with data and distributed training engineers to improve efficiency, reliability, and scalability across large GPU clusters.