Remote Distributed Pretraining Scientist: Protocol Learning

Pluralis Research; California; Remoto; USD 170.000 - 250.000; Tempo pieno

Vantaggi offerti da questo lavoro

Equity-heavy package

Remote-first culture

Visa sponsorship & relocation support

Open research problems & frontier work

Descrizione del lavoro

Pluralis Research is seeking a Research Scientist to architect and optimize large-scale, distributed pretraining systems for Protocol Learning. You’ll design data, tensor, and pipeline parallelism across heterogeneous GPUs and global regions, balancing efficiency, robustness, and convergence in real-world networks.

You will contribute production-quality code in PyTorch, work on fault tolerance, and build instrumentation to monitor throughput and model quality across hundreds of devices.

Competenze

• Experience training models across many devices using distributed training techniques.

Mansioni

• Distributed pretraining: implement and optimize model-parallel training across heterogeneous GPUs.
• Performance optimization to reduce communication overhead while preserving convergence.
• Elasticity and fault tolerance to handle node churn with robust checkpointing.

Conoscenze

Distributed training

PyTorch

Model parallelism

Production Python

Concurrency

Research code

Formazione

PhD or MSc in ML/CS

Strumenti

PyTorch

FSDP

DeepSpeed

Megatron

Descrizione del lavoro

Pluralis Research is seeking a Research Scientist to architect and optimize large-scale, distributed pretraining systems for Protocol Learning. You’ll design data, tensor, and pipeline parallelism across heterogeneous GPUs and global regions, balancing efficiency, robustness, and convergence in real-world networks.

You will contribute production-quality code in PyTorch, work on fault tolerance, and build instrumentation to monitor throughput and model quality across hundreds of devices.