ML Systems Engineer: Scalable Inference & Distributed Infra

Atlassian; Seattle (WA); Remote; USD 178,000 - 233,000; Full time

Benefits offered by this job

Health & wellbeing

Volunteer days

Equity opportunity

Job summary

Atlassian, the global software company, offers a flexible work model allowing office, home, or hybrid arrangements. Atlassians can work from anywhere with a legal entity, and this ML System Engineer role sits on the AI & ML Platform's Inference team, designing scalable model-serving infra.

The position requires 3+ years in software engineering with low-level systems and GPU inference experience, plus a track record in high-concurrency production serving.

Qualifications

• 3+ years of software engineering experience.
• Deep low-level systems programming in C/C++ or Rust.
• Experience with large-scale, high-concurrent production serving.
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems that serve billions of requests reliably.
• Benchmark, fine-tune, and accelerate inference engines.
• Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.
• Partner with senior ML engineers to fine-tune and deploy open-source LLMs.

Skills

3+ years software engineering

Low-level systems programming

High-concurrency production serving

GPU inference engines

Job description

Atlassian, the global software company, offers a flexible work model allowing office, home, or hybrid arrangements. Atlassians can work from anywhere with a legal entity, and this ML System Engineer role sits on the AI & ML Platform's Inference team, designing scalable model-serving infra.

The position requires 3+ years in software engineering with low-level systems and GPU inference experience, plus a track record in high-concurrency production serving.