Senior ML Inference Platform Engineer

Atlassian; Austin (TX); Hybrid; USD 206,000 - 269,000; Full time

Benefits offered by this job

Health and wellbeing resources

Paid volunteer days

Job summary

At Atlassian, the AI & ML Platform Team is seeking a Senior ML System Engineer to design and optimize large-scale model serving systems. You will own distributed infrastructure—from global KV caches to auto-scaling and low-level GPU kernel optimization—to enable production-grade AI workloads.

You will collaborate with senior ML engineers to fine‑tune open-source LLMs, implement robust CI/CD for inference deployments, and ensure high throughput with low latency under heavy traffic.

Qualifications

• 5+ years of software engineering experience
• 2+ years of system performance optimization experience
• Deep low-level systems programming in C/C++ or Rust
• Experience with large-scale, high-concurrent production serving
• Experience with GPU inference engines (Triton, TensorRT-LLM, vLLM, SGLang) and related optimizations

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache)
• Optimize latency and throughput of model inference under real production workloads
• Build reliable, high-concurrency serving systems that serve billions of requests reliably
• Benchmark, fine-tune, and accelerate inference engines
• Create robust CI/CD infrastructure for seamless model deployment and inference engine updates
• Partner with senior ML engineers to fine‑tune open-source LLMs and deploy

Skills

C/C++

Rust

Distributed systems

Low-latency optimizations

Performance analysis

Tools

Triton

TensorRT-LLM

vLLM

SGLang

Job description

At Atlassian, the AI & ML Platform Team is seeking a Senior ML System Engineer to design and optimize large-scale model serving systems. You will own distributed infrastructure—from global KV caches to auto-scaling and low-level GPU kernel optimization—to enable production-grade AI workloads.

You will collaborate with senior ML engineers to fine‑tune open-source LLMs, implement robust CI/CD for inference deployments, and ensure high throughput with low latency under heavy traffic.