Senior Staff, Multimodal Pretraining and Scaling

Hark; San Jose (CA); On-site; USD 180,000 - 450,000; Full time

Job summary

Hark is seeking a Member of Technical Staff in Pretraining Architecture and Scaling in San Jose. The Omni team builds the next generation of AI experiences beyond text, enabling models to understand and generate content across multiple modalities, including text, audio, and beyond.

You will work across the full pretraining stack to create natively multimodal models and set the base capabilities for downstream Omni experiences.

Qualifications

• Hands-on experience training large models.
• Strong experimental judgment in designing controlled experiments.
• Experience applying scaling laws to experiment design.
• Experience implementing and debugging distributed training workloads across GPU clusters.
• Record of improving model quality or training efficiency through research, engineering, open-source contributions, or production work.

Responsibilities

• Research model architecture, optimization, and scaling for large models.
• Establish baselines and run controlled experiments to evaluate architectural and optimization changes at scale.
• Develop training recipes, including learning-rate schedules, context lengths, token budgets, and compute allocation.
• Investigate mixture-of-experts architectures, hybrid attention, and long-context training.
• Diagnose loss spikes, divergence, and numerical issues with training engineers.

Skills

Large-model training

Experimental judgment

Scaling laws

Distributed training

Engineering skills

Open-source contributions

Tools

GPU clusters

Job description

Hark is seeking a Member of Technical Staff in Pretraining Architecture and Scaling in San Jose. The Omni team builds the next generation of AI experiences beyond text, enabling models to understand and generate content across multiple modalities, including text, audio, and beyond.

You will work across the full pretraining stack to create natively multimodal models and set the base capabilities for downstream Omni experiences.