Data Platform Engineer - Pretraining Pipelines (Remote)

poolside; Remote; GBP 70,000 - 90,000; Full time

Job summary

A leading AI technology company in the United Kingdom is seeking a core member for their Pretraining Data team. You will architect and maintain high-performance pipelines, transforming raw data into quality datasets for model training. The role demands a strong background in distributed data systems, expertise in Python, and a collaborative spirit. Benefits include fully remote work, generous vacation, health insurance, and a culture focused on well-being and inclusivity.

Benefits

Fully remote work & flexible hours

37 days/year of vacation & holidays

Health insurance allowance for you and dependents

Company-provided equipment

Home office allowances

Wellbeing and learning opportunities

Frequent team get-togethers

Inclusive people-first culture

Qualifications

• Experience in building production-grade, distributed data systems for machine learning.
• Strong algorithmic foundations.
• Ability to write clean and maintainable Python code.

Responsibilities

• Build and maintain high-performance pipelines for trillions of tokens.
• Deliver diverse and high-quality datasets for pre-training models.
• Collaborate with teams to ensure quality of the models delivered.

Skills

Distributed data systems

Orchestration: Slurm, Airflow, or Dagster

CI/CD, Grafana, Prometheus

Git, Docker, k8s, cloud services

Large-scale GPU clusters

Expert-level Python

Proficiency in Polars, Dask, or PySpark

Job description

A leading AI technology company in the United Kingdom is seeking a core member for their Pretraining Data team. You will architect and maintain high-performance pipelines, transforming raw data into quality datasets for model training. The role demands a strong background in distributed data systems, expertise in Python, and a collaborative spirit. Benefits include fully remote work, generous vacation, health insurance, and a culture focused on well-being and inclusivity.