Research Engineer, Data Infrastructure

Socket.dev; Palo Alto (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Competitive salary and equity

Medical/Dental/Vision coverage

401K with 6% matching

18 days PTO

$120/month for public transport

$120/month gym reimbursement

$400 monthly meal allowance

Visa sponsorship

BetterUp coaching

Job summary

A pioneering AI company in California is seeking a Data Infrastructure Engineer to build and operate large-scale data systems. The role involves architecting multi-cluster systems for optimized performance and maintaining modern storage solutions. Ideal candidates have over 4 years of experience in data infrastructure and proficiency in Python, along with a keen interest in MLOps. The position includes a competitive salary, healthcare coverage, and additional perks such as gym reimbursements and 401K matching.

Qualifications

• 4+ years of experience in Data Infrastructure, MLOps, or Infrastructure Engineering.
• Experience or strong interest in foundational compute and storage platforms.
• Proficient in Python and modern storage standards.

Responsibilities

• Build and scale distributed compute and storage systems.
• Architect and maintain multi-cluster orchestration layers.
• Help transition to modern storage formats.
• Contribute to the internal training platform development.
• Implement visibility and lineage systems for data pipelines.
• Manage cloud-native deployments for scalability.

Skills

Data Infrastructure expertise

MLOps knowledge

Python proficiency

Kubernetes tooling experience

Problem-solving in data lakes

Job description

About Mistral

At Mistral AI, we believe in the power of AI to simplify tasks, save time, and enhance learning and creativity. Our technology is designed to integrate seamlessly into daily working life.

We democratize AI through high-performance, optimized, open-source and cutting-edge models, products and solutions. Our comprehensive AI platform is designed to meet enterprise as well as personal needs. Our offerings include Le Chat, La Plateforme, Mistral Code and Mistral Compute - a suite that brings frontier intelligence to end-users.

We are a dynamic, collaborative team passionate about AI and its potential to transform society. Our diverse workforce thrives in competitive environments and is committed to driving innovation. Our teams are distributed between France, USA, UK, Germany and Singapore. We are creative, low-ego and team-spirited.

Join us to be part of a pioneering company shaping the future of AI. Together, we can make a meaningful impact. See more about our culture on https://mistral.ai/careers.

Role Summary

This role focuses on building and operating the next generation of data infrastructure at Mistral AI. You will be a core contributor to our evolution, helping us design and scale massive compute fleets and storage systems designed for high performance and scalability. You will help us move toward a future of decoupled control and data planes, scaling big data compute and storage platforms while ensuring secure and governed data access for MLOps and research. You will take full lifecycle ownership: from architecting the migration away from legacy orchestrators to implementing production-grade pipelines and participating in on-call rotations for critical training jobs.

What will you do

• Build & Scale: Help us reach our goal of operating massive distributed compute and storage systems
• Global Orchestration: Architect and maintain multi-cluster orchestration layers to optimize workload placement across diverse hardware and regions.
• Design Future-Proof Storage: Architect our transition to modern storage formats to handle fine-tuning datasets at a scale that anticipates exabyte growth.
• Platform Engineering: Contribute to the development of our internal training platform, ensuring seamless model training and fine-tuning capabilities across Kubernetes and SLURM based environments.
• Metadata & Lineage: Implement and manage systems to provide clear visibility and lineage as our data and model pipelines grow in complexity.
• Operational Excellence: Use modern deployment workflows to manage cloud-native deployments, ensuring our data platform can scale by o

About you

• Have 4+ years of experience in Data Infrastructure, MLOps, or Infrastructure Engineering.
• Have experience or a strong interest in supporting foundational compute and storage platforms.
• Are proficient in Python and enjoy solving the 'brittle data lake' problem with modern, columnar storage standards.
• Are well-versed in Kubernetes-native tooling and excited to debug large-scale distributed systems across multi-cluster environments.
• Take pride in building and operating scalable, reliable, and secure systems from the ground up.
• Are comfortable with ambiguity and the challenges of building high-scale infrastructure in a rapid-growth AI environment.

What we offer

• Competitive salary and equity.
• Healthcare: Medical/Dental/Vision covered for you and your family.
• Pension: 401K (6% matching)
• PTO: 18 days.
• Transportation: Reimburse office parking charges, or $120/month for public transport.
• Sport: $120/month reimbursement for gym membership.
• Meal stipend: $400 monthly allowance for meals (solution might evolve as we grow bigger).
• Visa sponsorship.
• Coaching: we offer BetterUp coaching on a voluntary basis.

By applying, you agree to our Applicant Privacy Policy.