Senior Staff Data Engineer, Web Data Pipelines

Avenue.io Inc.; Toronto; Remote; CAD 150,000 - 190,000; Full time

Benefits offered by this job

Weekly lunch stipend 75 USD/75 EUR or

Full health and dental benefits

RRSP matching / 401K / Pension

Parental Leave top-up up to 6 months

Education stipend for conferences, etc

6 weeks paid vacation

Home office stipend

Job summary

Cohere is seeking a Senior Member of Technical Staff specializing in web data for pre-training. You will design and own the large-scale web data pipeline that feeds Cohere’s language models, transforming raw internet data into high-quality training data.

You will work with extraction, parsing, deduplication, and filtering components and collaborate with data and evaluation teams to optimize corpus quality. The role is remote-friendly with offices in major cities; strong collaboration across

Qualifications

• Strong software engineering skills with Python and experience building data pipelines.
• Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools.
• Experience working with large-scale web datasets.
• Knowledge of data quality assessment techniques and experimentation with data mixtures.
• A passion for bridging research and engineering to solve data-related AI model-training challenges.

Responsibilities

• Maintain large-scale pipelines for processing web corpora.
• Develop filtering and quality-scoring systems for high-value web documents.
• Analyze web data composition across domains, languages and time periods.
• Develop and maintain deduplication pipelines.
• Collaborate with researchers and engineers to meet data pipeline needs for language models.

Skills

Python

Data pipelines

Large-scale web data

Data quality assessment

Research & engineering collaboration

Tools

Apache Spark

Apache Beam

Pandas

Job description

Cohere is seeking a Senior Member of Technical Staff specializing in web data for pre-training. You will design and own the large-scale web data pipeline that feeds Cohere’s language models, transforming raw internet data into high-quality training data.

You will work with extraction, parsing, deduplication, and filtering components and collaborate with data and evaluation teams to optimize corpus quality. The role is remote-friendly with offices in major cities; strong collaboration across