Data Infrastructure Engineer for ML Pipelines

Luma; Redwood City (CA); On-site; USD 150,000 - 240,000; Full time

Job summary

Luma is seeking a Data Infrastructure Engineer in our Research group to build and scale data pipelines for multimodal AI research and internal ML platforms. You will work closely with ML researchers and product teams to design reliable, high-performance data systems that accelerate experimentation and deployment.

The role emphasizes supporting complex ML workflows, not traditional product data infra, with a focus on scalable pipelines, monitoring, and cross-functional collaboration.

Qualifications

• Experience with high-throughput data infrastructure for ML workflows.
• Strong understanding of distributed systems and data engineering at scale.
• Ability to design and optimize data pipelines for ML research and internal teams.

Responsibilities

• Build and maintain scalable data infrastructure for high-throughput machine learning workflows
• Collaborate with ML researchers and product teams to ensure data systems meet evolving needs
• Develop and optimize large-scale data pipelines and batch processing jobs
• Contribute to the architecture and implementation of reliable, high-performance data platforms
• Integrate open-source tools and continuously improve data infrastructure through monitoring and tuning
• Participate in cross-functional projects to improve data reliability, scalability, and operational excellence
• Support the evaluation and adoption of new programming languages and frameworks relevant to data infrastructure
• Engage in continuous improvement of data infrastructure through monitoring, troubleshooting, and performance tuning
• Collaborate with research & engineering teams to help define and refine best practices for data infrastructure development

Skills

Python

Distributed systems

ML data pipelines

Data engineering

Collaboration

Big data

Tools

Ray

Spark

Beam

Airflow

Job description

Luma is seeking a Data Infrastructure Engineer in our Research group to build and scale data pipelines for multimodal AI research and internal ML platforms. You will work closely with ML researchers and product teams to design reliable, high-performance data systems that accelerate experimentation and deployment.

The role emphasizes supporting complex ML workflows, not traditional product data infra, with a focus on scalable pipelines, monitoring, and cross-functional collaboration.