ML Inference Systems Engineer – Remote, Scalable, Low-Latency

Atlassian Corp.; Seattle, Northern (WA, KY); Hybrid; USD 178,000 - 233,000; Full time

Benefits offered by this job

Health and wellbeing resources

Volunteer days

Job summary

At Atlassian, the ML System Engineer will design and optimize large-scale model serving systems, spanning distributed infrastructure to low-level GPU kernel optimizations. You’ll own end-to-end components from caching and batching to auto-scaling and deployment.

The role emphasizes building reliable, high-concurrency serving systems, benchmarking and tuning inference engines, and partnering with senior ML engineers to deploy open-source LLMs.

Qualifications

• 3+ years of software engineering experience.
• Deep low-level systems programming in C/C++ or Rust.
• Experience with large-scale, high-concurrent production serving.
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).

Responsibilities

• Architect scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling).
• Optimize latency and throughput under production workloads.
• Build high-concurrency serving systems that handle billions of requests.
• Benchmark, fine-tune, and accelerate inference engines.

Skills

System programming

Distributed systems

GPU inference engines

High-concurrency serving

C/C++ or Rust

Tools

GPU frameworks

Job description

At Atlassian, the ML System Engineer will design and optimize large-scale model serving systems, spanning distributed infrastructure to low-level GPU kernel optimizations. You’ll own end-to-end components from caching and batching to auto-scaling and deployment.

The role emphasizes building reliable, high-concurrency serving systems, benchmarking and tuning inference engines, and partnering with senior ML engineers to deploy open-source LLMs.