Mid Training

xAI; Palo Alto (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health insurance

Life and AD&D insurance

Fertility benefits

Visa sponsorship

401(k) plan

Job summary

xAI is seeking a hands-on ML data engineer to scale data pipelines and verify large-scale models in the Bay Area. The role focuses on advancing synthetic data, model distillation, and robust evaluation across mid-training checkpoints.

You will work with cutting-edge frameworks and collaborate across teams to push capabilities in a fast-paced environment. Candidates should be located near the Bay Area or be open to relocation.

Qualifications

• Strong engineering abilities in Spark, Ray, and other frameworks for large-scale data processing.
• Expertise in ML and large model scaling, with familiarity across scaling laws.
• Strong ability to design ML experiments.
• Familiarity with state-of-the-art techniques for curating AI training data for text, image, audio, and video modalities.

Responsibilities

• Scale synthetic coding data to trillions of tokens with large-scale docker verification.
• Distill the intelligence of flagship models into flash models through synthetic data generation.
• Optimize mid-training data mixtures to boost the ceiling for RL.
• Engineer long-context data recipes.
• Develop robust and diverse evaluation for mid-training checkpoints.

Skills

Spark

Ray

Large-scale data processing

ML experimentation design

Data curation for AI training

Tools

Docker

Job description

• Scale synthetic coding data to trillions of tokens with large-scale docker verification
• Distill the intelligence of flagship models into flash models through synthetic data generation
• Optimize mid-training data mixtures to boost the ceiling for RL
• Engineer long-context data recipes
• Develop robust and diverse evaluation for mid-training checkpoints

Benefits

• Health and wellness: Comprehensive health insurance including medical, dental, vision, and disability coverage
• Life and family: Life and AD&D insurance and fertility benefits to ensure our team’s well-being and peace of mind
• Flexible vacation: We work hard but avoid burn out. Take time off when you need it
• Visa sponsorship: We support international talent with visa sponsorship to join our team
• 401(k) plan: Retirement savings plan to secure your financial future

The role is based in the Bay Area [San Francisco and Palo Alto]. Candidates are expected to be located near the Bay Area or open to relocation.

• Strong engineering abilities in Spark, Ray, and other frameworks for large-scale data processing
• Expertise in ML and large model scaling, with familiarity across all kinds of scaling laws
• Strong ability to design ML experiments
• Familiarity with state-of-the-art techniques for curating AI training data for text, image, audio, and video modalities