Senior Web Data Engineer—Large-Scale AI Pretraining

Cohere; United States; On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Inclusive culture

Weekly lunch stipend

Full health and dental benefits

100% Parental Leave top-up

6 weeks of vacation

Job summary

Cohere is looking for a Senior Member of Technical Staff specializing in web data to develop pipelines for their advanced language models. This role involves handling large-scale web data, ensuring quality and relevance for AI training. Candidates should have a strong background in software engineering and data processing frameworks.

The position is remote-friendly, allowing flexibility with no location restrictions. Cohere values diversity and inclusivity, offering a vibrant workplace and competitive benefits.

Qualifications

• Strong software engineering skills with a focus on Python.
• Experience in maintaining large-scale data pipelines.
• Ability to analyze web data composition and quality.

Responsibilities

• Develop large-scale web data pipelines for model training.
• Implement filtering and quality-scoring systems.
• Collaborate with researchers and engineers on data pipelines.

Skills

Proficiency in Python

Experience building data pipelines

Familiarity with data processing frameworks (e.g., Apache Spark)

Experience with large-scale web datasets

Knowledge of data quality assessment techniques

Job description

Cohere is looking for a Senior Member of Technical Staff specializing in web data to develop pipelines for their advanced language models. This role involves handling large-scale web data, ensuring quality and relevance for AI training. Candidates should have a strong background in software engineering and data processing frameworks.

The position is remote-friendly, allowing flexibility with no location restrictions. Cohere values diversity and inclusivity, offering a vibrant workplace and competitive benefits.