Remote Pretraining Data Engineer - High-Scale ML Pipelines

Jaide Health; United States; On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Fully remote work

Flexible hours

37 days/year of vacation & holidays

Health insurance allowance

Company-provided equipment

Wellbeing allowances

Frequent team get togethers

Job summary

Jaide Health is looking for a core member of the Pretraining Data team to build and scale their Model Factory. This key role focuses on architecting high-performance pipelines for transforming vast data into high-quality datasets for artificial intelligence and coding agents. Responsibilities include maintaining distributed systems that handle petabyte-scale data and collaborating with various teams to enhance model performance. The position offers fully remote work, 37 days of vacation, and a supportive culture.

Qualifications

• Strong background in building production-grade distributed data systems for machine learning.
• Ability to write clean and maintainable code in Python.
• Experience with orchestration tools such as Slurm or Airflow.

Responsibilities

• Build and maintain high-performance pipelines for trillions of tokens.
• Deliver diverse and high-quality datasets for pre-training foundation models.
• Collaborate with other teams to ensure model quality.

Skills

Distributed data systems for machine learning

Expert-level python knowledge

Strong algorithmic foundations

Proficiency with libraries like Polars, Dask, or PySpark

Tools

Slurm

Airflow

Docker

k8s

cloud managed services

Job description

Jaide Health is looking for a core member of the Pretraining Data team to build and scale their Model Factory. This key role focuses on architecting high-performance pipelines for transforming vast data into high-quality datasets for artificial intelligence and coding agents. Responsibilities include maintaining distributed systems that handle petabyte-scale data and collaborating with various teams to enhance model performance. The position offers fully remote work, 37 days of vacation, and a supportive culture.