Remote Senior ML Systems Engineer - Model Serving

Atlassian; Northern (KY); Hybrid; USD 149,000 - 235,000; Full time

Benefits offered by this job

Benefits and bonuses

Equity

Job summary

Atlassian is seeking a Senior ML System Engineer to design and optimize large-scale model serving systems across distributed infrastructure. You will own everything from global KV caches and auto-scaling to low-level kernel optimizations for high-concurrency production workloads.

The role requires deep expertise in C/C++ or Rust, experience with GPU inference engines, and a track record of optimizing latency and throughput in production.

Qualifications

• 5+ years of software engineering experience, 2+ years of system performance optimization.
• Deep low-level systems programming in C/C++ or Rust.
• Experience with large-scale, high-concurrency production serving.
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM).

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems that serve billions of requests reliably
• Benchmark, fine-tune, and accelerate inference engines.
• Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.
• Partner with senior ML engineers to fine‑tune open-source LLMs and deploy

Skills

C/C++ or Rust

Distributed systems

GPU inference

Performance optimization

CI/CD for inference

Tools

vLLM

SGLang

Triton

TensorRT-LLM

Job description

Atlassian is seeking a Senior ML System Engineer to design and optimize large-scale model serving systems across distributed infrastructure. You will own everything from global KV caches and auto-scaling to low-level kernel optimizations for high-concurrency production workloads.

The role requires deep expertise in C/C++ or Rust, experience with GPU inference engines, and a track record of optimizing latency and throughput in production.