ML Data Engineer - Pretraining Pipelines (Remote)

Cohere; New York (NY); Remote; USD 140,000 - 190,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

Retirement plan matching

Parental leave

Learning stipend for conferences andco

Vacation and enrichment benefits

Offsite travel budget

Home office stipend

Job summary

Cohere is a leading security-first enterprise AI company crafting foundation models and end-to-end products for real-world problems. We are seeking a Machine Learning Engineer focused on pretraining data to build robust data pipelines, perform data ablations, and create effective data mixtures that improve model performance and throughput.

You will blend research and engineering, curating diverse data including multilingual corpora, and collaborating with researchers to optimize training

Qualifications

• Strong software engineering skills with Python and data pipelines.
• Familiarity with curriculum learning, data mixing and data attribution.
• Familiarity with Spark, Beam, Pandas or similar tools.
• Experience with large-scale datasets including web data, code data, multilingual corpora.
• Knowledge of data quality assessment techniques and data mixtures.
• Passion for bridging research and engineering to solve data-related AI training challenges.
• Bonus: papers at top-tier venues (e.g., NeurIPS, ICML, ICLR, ACL).

Responsibilities

• Conduct data ablations to assess data quality and experiment with data mixtures to enhance model performance.
• Develop robust data modeling techniques for structured datasets to optimize training efficiency.
• Research and implement innovative data curation methods leveraging Cohere’s infrastructure.
• Collaborate with researchers and engineers to ensure data pipelines support cutting-edge language models.

Skills

Python

Data pipelines

Software engineering

Curriculum learning

Data mixing

Data attribution

Large-scale data

Multilingual corpora

Data quality assessment

Tools

Apache Spark

Apache Beam

Pandas

Job description

Cohere is a leading security-first enterprise AI company crafting foundation models and end-to-end products for real-world problems. We are seeking a Machine Learning Engineer focused on pretraining data to build robust data pipelines, perform data ablations, and create effective data mixtures that improve model performance and throughput.

You will blend research and engineering, curating diverse data including multilingual corpora, and collaborating with researchers to optimize training