Lead Data Engineer for Multimodal ML Pipelines

Sarah Smith Fund; United States; Hybrid; USD 150,000 - 230,000; Full time

Benefits offered by this job

Competitive salary

Equity

Health benefits

401k matching

Growth opportunities

Hybrid on-site/remote

Job summary

Pika is hiring a Staff/Lead Research Engineer, Data to architect and scale data pipelines that support model training for multimodal foundation models. You will own data ingestion, labeling, filtering, and retrieval, collaborating with research teams to curate diverse datasets and ensure data quality and compliance.

This role focuses on production-ready ML data infrastructure, privacy and ethics throughout the data lifecycle, and staying ahead of data management advances to power real-time

Qualifications

• 5+ years of experience building and scaling data pipelines for ML applications.
• Experience with ML data curation for LLMs, VLMs, or other large-scale multimodal models.
• Strong background in distributed data systems (e.g., Spark, Hadoop, Ray) and ETL workflows.
• Proven ability to build robust, scalable, production-grade data infrastructure for ML pipelines.
• Experience developing tools for data labeling, filtering, deduplication, quality assurance, and dataset management.
• Strong programming skills (Python, SQL, PySpark) and familiarity with cloud data platforms (AWS, GCP, Azure).
• Knowledge of privacy, compliance, ethics, and best practices in data collection and management.

Responsibilities

• Take ownership of large-scale data pipeline architecture and implementation to support model training and research workflows for text, image, audio, and video datasets.
• Partner with research and engineering teams to curate, clean, and manage diverse, sensory-rich datasets for pre-training and mid-training of multimodal models.
• Develop strategies and tools for scalable data ingestion, labeling, filtering, augmentation, and storage.
• Ensure data quality, reliability, and compliance, including managing privacy and ethical considerations throughout the data lifecycle.
• Optimize data processing, transformation, and delivery for large-scale distributed training pipelines.
• Prototype and productionize new methods for dataset creation, management, and continuous improvement in response to researcher needs.
• Contribute to the integration of research-driven data advancements into production-ready systems.
• Stay informed on emerging data engineering and ML data management developments, bringing best practices to our systems.

Skills

Python

SQL

PySpark

ETL workflows

Data pipelines

ML data curation

Cross-functional collaboration

Data quality

Tools

Spark

Hadoop

Ray

AWS

GCP

Azure

Job description

Pika is hiring a Staff/Lead Research Engineer, Data to architect and scale data pipelines that support model training for multimodal foundation models. You will own data ingestion, labeling, filtering, and retrieval, collaborating with research teams to curate diverse datasets and ensure data quality and compliance.

This role focuses on production-ready ML data infrastructure, privacy and ethics throughout the data lifecycle, and staying ahead of data management advances to power real-time