Senior ML Systems Engineer - Scalable Inference

Atlassian; Seattle (WA); Hybrid; USD 206,000 - 269,000; Full time

Benefits offered by this job

Health and wellbeing resources

Paid volunteer days

Job summary

At Atlassian, we are seeking a Senior ML System Engineer on the AI & ML Platform’s Inference Team. You will design and optimize large-scale model serving systems end-to-end, owning distributed infrastructure and low-level optimizations from GPU kernels to speculative decoding.

You will architect scalable model-serving infrastructure, reduce latency, and deploy high-concurrency serving that handles billions of requests, while building CI/CD workflows for model updates and collaborating with

Qualifications

• 5+ years of software engineering experience, 2+ years of system performance optimization experience.
• Deep low-level systems programming (C/C++ or Rust).
• Experience with large-scale, high-concurrent production serving.
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).
• Strong background in system optimizations: batching, caching, load balancing, parallelism.

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems that serve billions of requests reliably
• Benchmark, fine-tune, and accelerate inference engines.
• Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.
• Partner with senior ML engineers to fine‑tune open-source LLMs and deploy

Skills

Software engineering

Low-level systems programming

Large-scale serving

GPU inference

System optimizations

Tools

GPU inference engines

Job description

At Atlassian, we are seeking a Senior ML System Engineer on the AI & ML Platform’s Inference Team. You will design and optimize large-scale model serving systems end-to-end, owning distributed infrastructure and low-level optimizations from GPU kernels to speculative decoding.

You will architect scalable model-serving infrastructure, reduce latency, and deploy high-concurrency serving that handles billions of requests, while building CI/CD workflows for model updates and collaborating with