Senior ML Systems Engineer: Scalable Model Serving & GPUs

Atlassian; San Francisco (CA); Hybrid; USD 206,000 - 269,000; Full time

Benefits offered by this job

Health and wellbeing resources

Paid volunteer days

Benefits information

Job summary

At Atlassian, a Senior ML System Engineer on the AI & ML Platform team will design and optimize large-scale model serving systems end-to-end, handling distributed infrastructure and low-level optimizations for production workloads.

The role requires deep expertise in C/C++ or Rust, GPU inference, and scalable, high-concurrency serving, with collaboration on CI/CD pipelines and deployment of open-source LLMs.

Qualifications

• 5+ years of software engineering experience.
• 2+ years of system performance optimization experience.
• Deep low-level systems programming (C/C++ or Rust).
• Experience with large-scale, high-concurrent production serving.

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems that serve billions of requests reliably.
• Benchmark, fine-tune, and accelerate inference engines.
• Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.
• Partner with senior ML engineers to fine‑tune open-source LLMs and deploy

Skills

Software engineering

System performance optimization

Low-level systems programming

Tools

vLLM, SGLang, Triton, TensorRT-LLM

Job description

At Atlassian, a Senior ML System Engineer on the AI & ML Platform team will design and optimize large-scale model serving systems end-to-end, handling distributed infrastructure and low-level optimizations for production workloads.

The role requires deep expertise in C/C++ or Rust, GPU inference, and scalable, high-concurrency serving, with collaboration on CI/CD pipelines and deployment of open-source LLMs.