Distributed Pretraining Scientist for Frontier Models

OpenKrill; United States; Remoto; USD 180.000 - 240.000; Tempo pieno

Vantaggi offerti da questo lavoro

Equity-Heavy Package

Remote-First Culture

Visa Sponsorship & Relocation support

Open Problems: frontier model research

Descrizione del lavoro

Pluralis Research seeks a Research Scientist to build and scale training systems for Protocol Learning across heterogeneous hardware and regions, connected by ordinary internet. You will tackle distributed pretraining, fault tolerance, and performance optimizations while advancing frontier models.

We value hands-on experience with large-scale distributed setups, production-grade Python, and evidence of open-source or research-oriented projects.

Competenze

• Hands-on distributed training on many devices using PyTorch with model-parallelism.
• Strong engineering practices in Python, concurrency and failure handling.
• Evidence of shipped systems or open-source projects is a plus.

Mansioni

• Distributed pretraining: implement and optimize model-parallel training.
• Performance optimization to reduce communication overhead.
• Elasticity and fault tolerance for churn and node failures.
• Run instrumentation to monitor throughput and model quality.

Conoscenze

Distributed training across devices

PyTorch

Model parallelism

Production-quality Python

Strumenti

FSDP / DeepSpeed / Megatron

Descrizione del lavoro

Pluralis Research seeks a Research Scientist to build and scale training systems for Protocol Learning across heterogeneous hardware and regions, connected by ordinary internet. You will tackle distributed pretraining, fault tolerance, and performance optimizations while advancing frontier models.

We value hands-on experience with large-scale distributed setups, production-grade Python, and evidence of open-source or research-oriented projects.