Remote Data Engineer - Pretraining Pipelines

Poolside; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Fully remote work & flexible hours

37 days/year of vacation & holidays

Health insurance allowance for you &;y

Company-provided equipment

Well-being, always-be-learning & home/

Frequent team get togethers

Diverse & inclusive people-first

Job summary

Poolside is building data foundations for large language models. You will join the Pretraining Data team to architect and maintain high-performance pipelines that transform trillions of raw tokens into the high-quality dataset fuel our models require.

This hands-on, fully remote role emphasizes scalable ingestion, deduplication, and streaming systems, cross-team collaboration, and a culture of learning, experimentation, and AGI-oriented research.

Qualifications

• Building production-grade, distributed data systems for ML
• Expert-level Python programming with clean, maintainable code
• Strong algorithmic foundations for data processing
• Experience with large-scale data tools and libraries (e.g., Polars, Dask, PySpark)

Responsibilities

• Build and maintain high-performance pipelines for trillions of tokens
• Deliver diverse and high-quality datasets for pre-training foundation models
• Collaborate with Pretraining, Posttraining, Evals and Product to ensure model quality

Skills

Distributed data systems

Python programming

Algorithms

Code quality

Tools

Polars

Dask

PySpark

Job description

Poolside is building data foundations for large language models. You will join the Pretraining Data team to architect and maintain high-performance pipelines that transform trillions of raw tokens into the high-quality dataset fuel our models require.

This hands-on, fully remote role emphasizes scalable ingestion, deduplication, and streaming systems, cross-team collaboration, and a culture of learning, experimentation, and AGI-oriented research.