Research Engineer - Pre-Training

Pluralis Research

Port Macquarie

Remote

AUD 180,000 - 280,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity-Heavy Package
Remote-First Culture
Visa sponsorship & relocation support
Open problems: frontier research

Job summary

Pluralis Research is seeking a Research Engineer to build the training system that scales Protocol Learning from the 8B run to frontier scale: large models on heterogeneous hardware across regions, connected via ordinary internet.

You will contribute to distributed pretraining, performance optimization, elasticity, and run instrumentation, with a remote-first culture and Visa sponsorship options. This role requires hands-on distributed training experience and strong engineering skills.

Qualifications

  • Trained models across many devices in PyTorch with FSDP, DeepSpeed, Megatron, or own implementation.
  • Ability to design data, tensor, and pipeline parallelism.
  • Production-quality Python and concurrency handling.
  • Experience shipping systems or open-source projects.

Responsibilities

  • Distributed pretraining: Implement and optimize model-parallel training for large models on heterogeneous GPUs.
  • Performance optimization: Reduce communication overhead while preserving convergence in challenging network environments.
  • Elasticity and fault tolerance: Handle node churn with robust checkpointing and synchronization.
  • Run instrumentation: Build monitoring for throughput, bottlenecks, and model quality across devices.

Skills

Hands-on distributed training
Strong engineering
Evidence of execution
Mission alignment

Tools

PyTorch
FSDP
DeepSpeed
Megatron

Job description

Job Description

Pluralis Research works on Protocol Learning: training and serving large models in a fully decentralized way on small consumer-grade devices connected via the internet. Despite being dismissed as infeasible, we have made significant advances on this problem, most recently Agora, a permissionless run that pretrained an 8B model from scratch on consumer GPUs spread over the internet, with no single participant ever holding the full weights (tech report). While many of the core research problems have been solved, Protocol Learning unlocks a series of new challenges. For the mission in full, read A Third Path: Protocol Learning.

This setting breaks nearly every assumption of datacenter training: communication-efficient training across different parallelism axes, fault tolerance as nodes join and drop mid-run, heterogeneous compute and networks, and robustness to malicious participants. Our published methods include Subspace Networks, Factored Gossip DiLoCo, AsyncMesh, and Sentinel.

As a Research Engineer you'll build the training system that takes Protocol Learning from the 8B run to frontier scale: large models on heterogeneous hardware, in physically different regions, connected by ordinary internet.

Key Responsibilities
  • Distributed pretraining : Implement and optimize model-parallel training. Data, pipeline, and tensor parallelism for large models on heterogeneous GPUs under low-bandwidth, high-latency links.

  • Performance optimization : Implement techniques that reduce communication overhead while maintaining model convergence in challenging network environments.

  • Elasticity and fault tolerance : Make runs survive node churn. Robust checkpointing, state synchronization, and recovery as participants join and leave.

  • Run instrumentation : Build the monitoring that shows throughput, bottlenecks, and model quality across hundreds of devices.

What We're Looking For
  • Hands-on distributed training (required) : You've trained models across many devices in PyTorch with FSDP, DeepSpeed, Megatron, or your own implementation. You understand data, tensor, and pipeline parallelism.

  • Strong engineering : Production-quality Python. Concurrency, failure handling, profiling before optimizing.

  • Evidence of execution : Shipped systems, research code, open-source work, or serious personal projects.

  • Mission alignment : You believe Protocol Learning is the viable third path for collective, trustless, and sovereign AI.

Nice to Have
  • Hands-on experience training or serving large language models such as Nemotron, Qwen or OLMo.

  • Experience with P2P networking and NAT traversal.

  • Experience with post-training and RL.

  • Experience with inference and serving systems.

  • Experience at proprietary, open-weight and open-source AI labs

Compensation & Benefits
  • Equity-Heavy Package : We offer significant ownership for key technical contributors in addition to a high base salary.

  • Remote-First Culture : Flexible work environment with team members distributed globally.

  • Visa Sponsorship : Optional full visa sponsorship and relocation support to either Australia or the US.

  • Open Problems : Training and serving frontier models on hardware you don't control, over networks you don't own, mostly has no published answers yet. You'll write some of the first ones.

FYI's
  • We work remotely across the world, with the main teams in Australia and North America. You'll need to be comfortable working across timezones.

  • Applicants must have professional-level English proficiency (written and spoken).

We are backed by Union Square Ventures and other tier-1 investors, and we are a world-class, deeply technical team of ML researchers. Pluralis is unapologetically ideological. We believe AI, and the world, end up on a better path if we succeed in implementing the protocol for intelligence. If this resonates,

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Research Engineer - Pre-Training
Research Engineer - Pre-Training

Pluralis Research • Mackay Regional

Remote
AUD 180,000 - 250,000
Equity-Heavy Package
Remote-First Culture
Visa Sponsorship & Relocation
Research Engineer - Pre-Training
Research Engineer - Pre-Training

Pluralis Research • Hobart

Remote
AUD 150,000 - 210,000
Equity-heavy package
Remote-friendly culture
Visa sponsorship
+1
Research Engineer - Post-Training
Research Engineer - Post-Training

Pluralis Research • Townsville City

Remote
AUD 140,000 - 210,000
Equity-Heavy Package
Remote-First Culture
Visa Sponsorship
+1
Research Engineer - Post-Training
Research Engineer - Post-Training

Pluralis Research • Bathurst

Remote
AUD 180,000 - 260,000
Equity ownership
Remote-first culture
Visa sponsorship & relocation support
Distributed ML Systems Engineer: Protocol Learning (Remote)
Distributed ML Systems Engineer: Protocol Learning (Remote)

Pluralis Research • Port Macquarie

Remote
AUD 180,000 - 280,000
Equity-Heavy Package
Remote-First Culture
Visa sponsorship & relocation support
+1
Remote RL Post-Training Engineer for Decentralized AI
Remote RL Post-Training Engineer for Decentralized AI

Pluralis Research • Townsville City

Remote
AUD 140,000 - 210,000
Equity-Heavy Package
Remote-First Culture
Visa Sponsorship
+1
Research Engineer - Decentralized RL Post-Training
Research Engineer - Decentralized RL Post-Training

Pluralis Research • Bathurst

Remote
AUD 180,000 - 260,000
Equity ownership
Remote-first culture
Visa sponsorship & relocation support
Trustless AI Verifier for Decentralized Training
Trustless AI Verifier for Decentralized Training

Pluralis Research • Australia

Remote
AUD 120,000 - 180,000
Equity-Heavy Package
Remote-First Culture
Visa Sponsorship & Relocation
ML Researcher
ML Researcher

Base Compute • Melbourne

On-site
AUD 150,000 - 210,000
Equity
Hybrid work (occasional in-office)
Direct influence on technical方向
NLP Engineer
NLP Engineer

Jobgether • Australia

Remote
AUD 120,000 - 180,000
Fully remote work
Competitive compensation
28 days paid leave
+9
Top locations