Remote Distributed Training Engineer - Frontier Protocol Learning

Pluralis-Research; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Equity-Heavy Package

Remote-First Culture

Visa Sponsorship

Relocation support

Job summary

Pluralis Research is pursuing Protocol Learning to enable training and serving large models on consumer hardware across the internet. The team works on decentralised training methods and enables a permissionless run that scales beyond traditional datacenters.

The Research Engineer role focuses on building the training system to scale from 8B models to frontier-scale architectures, handling heterogeneous hardware and cross-region deployments while remaining remote-first and globally distributed.

Qualifications

• Hands-on distributed training across many devices in PyTorch with FSDP/DeepSpeed/Megatron.
• Strong engineering with production-quality Python and concurrency handling.
• Evidence of execution: shipped systems, research code, open-source projects.
• Mission-aligned with Protocol Learning as a viable AI path.

Responsibilities

• Distributed pretraining: implement and optimize model-parallel training across heterogeneous GPUs.
• Performance optimization to reduce communication overhead while preserving convergence.
• Elasticity and fault tolerance: robust checkpointing and state synchronization as nodes join/leave.
• Run instrumentation: monitor throughput and bottlenecks across hundreds of devices.

Skills

Distributed training

Python engineering

Execution evidence

Mission alignment

Tools

FSDP

DeepSpeed

Megatron

Job description

Pluralis Research is pursuing Protocol Learning to enable training and serving large models on consumer hardware across the internet. The team works on decentralised training methods and enables a permissionless run that scales beyond traditional datacenters.

The Research Engineer role focuses on building the training system to scale from 8B models to frontier-scale architectures, handling heterogeneous hardware and cross-region deployments while remaining remote-first and globally distributed.