Senior Web Data Engineer for AI Pre-Training Pipelines

Cohere; España; On-site; PHP 7,384,615 - 9,230,769; Full time

Benefits offered by this job

Open and inclusive culture

Weekly lunch stipend

Full health and dental benefits

6 weeks of paid vacation

Remote-friendly work environment

Annual company offsite

Job summary

Cohere is seeking a Senior Member of Technical Staff specializing in web data to develop a large-scale web data pipeline for advanced language models.

Your key role will involve transforming noisy internet data into high-quality training data essential for AI systems. Collaborate with various teams to enhance the quality of language understanding and generation capabilities.

This role is remote-friendly, allowing flexibility in work location.

Qualifications

• Strong software engineering skills with proficiency in Python.
• Experience building data pipelines is essential.
• Familiarity with data processing frameworks is required.

Responsibilities

• Maintain large-scale pipelines for processing web corpora.
• Work on filtering and quality-scoring systems.
• Analyze web data composition across various parameters.

Skills

Software engineering skills

Proficiency in Python

Experience building data pipelines

Familiarity with Apache Spark

Experience with large-scale web datasets

Tools

Apache Beam

Pandas

Job description

Cohere is seeking a Senior Member of Technical Staff specializing in web data to develop a large-scale web data pipeline for advanced language models.

Your key role will involve transforming noisy internet data into high-quality training data essential for AI systems. Collaborate with various teams to enhance the quality of language understanding and generation capabilities.

This role is remote-friendly, allowing flexibility in work location.