Staff ML Engineer — Pretraining Data & Pipelines

Cohere; United States; Remote; USD 140,000 - 200,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP/401K matching

Parental leave

Education stipend

Vacation 6 weeks

Traveling to other offices

Home office stipend

Job summary

Cohere is seeking a Machine Learning Engineer focused on pretraining data to build data pipelines for advanced language models. You will conduct ablations to evaluate data quality and create pre-training data mixtures that improve throughput and accelerator utilization.

The role blends research and engineering to advance data-driven AI training, with remote-friendly options and offices worldwide. You will collaborate with researchers and engineers across locations to drive model performance and

Qualifications

• Strong software engineering skills with Python and data pipelines.
• Familiarity with curriculum learning, data mixing and data attribution.
• Familiarity with data processing frameworks such as Spark, Beam, Pandas or similar tools.
• Experience working with large-scale datasets including web data, code data and multilingual corpora.
• Knowledge of data quality assessment techniques and experimentation with data mixtures.
• A passion for bridging research and engineering to solve data-related AI model training challenges.

Responsibilities

• Conduct data ablations to assess data quality and experiment with data mixtures to enhance model performance.
• Develop robust data modeling techniques to ensure datasets are structured for training efficiency.
• Research and implement innovative data curation methods leveraging Cohere’s infrastructure.
• Collaborate with researchers and engineers to ensure data pipelines meet model requirements.

Skills

Python

Data pipelines

Curriculum learning

Data mixing

Data attribution

Spark/Beam/Pandas

Large-scale datasets

Tools

Apache Spark

Apache Beam

Pandas

Job description

Cohere is seeking a Machine Learning Engineer focused on pretraining data to build data pipelines for advanced language models. You will conduct ablations to evaluate data quality and create pre-training data mixtures that improve throughput and accelerator utilization.

The role blends research and engineering to advance data-driven AI training, with remote-friendly options and offices worldwide. You will collaborate with researchers and engineers across locations to drive model performance and