Senior Machine Learning Systems Engineer

Atlassian; Amsterdam; Hybrid; EUR 120,000 - 180,000; Full time

Benefits offered by this job

Health resources

Volunteer days

Community engagement

Job summary

Atlassian's AI & ML Platform is seeking a Senior ML System Engineer to design and optimize large-scale model serving systems end-to-end. You’ll own distributed infrastructure, from global KV cache to batching, and will work on low-level GPU kernels and quantization.

On day one, expect 5+ years in software engineering, 2+ years in performance optimization, strong C/C++ or Rust, and experience with high-concurrency production serving and GPU inference engines.

Qualifications

• 5+ years of software engineering experience with system-level focus.
• Deep expertise in low-level systems programming and performance tuning.
• Experience with large-scale model serving and high concurrency environments.

Responsibilities

• Architect scalable distributed infrastructure for model serving, including load balancing and auto-scaling.
• Optimize latency and throughput under production workloads.
• Build reliable, high-concurrency serving systems handling billions of requests.
• Benchmark, tune, and accelerate inference engines.
• Create robust CI/CD pipelines for deployment and updates.
• Collaborate with senior ML engineers to fine-tune open-source LLMs.

Skills

Distributed systems

Low-level programming

Performance optimization

GPU inference

High concurrency

Tools

C/C++

Rust

vLLM

SGLang

Triton

TensorRT-LLM

Job description

Working at Atlassian

Atlassians can choose where they work - whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity.

• Full-Time

About the Central AI Org

Our organization is dedicated to driving AI innovation across all Atlassian products and platforms. We aim to deliver seamless AI experiences while establishing a robust Atlassian AI infrastructure for the future. Our purpose is to:

• Develop horizontal AI capabilities and infrastructure that can be leveraged across all products.
• Establish a centralized Search, Q&A, and Conversational AI system that integrates seamlessly with all Atlassian products.
• Explore integrating Atlassian products with AI solutions beyond the Atlassian ecosystem.

About the AI & ML Platform Team

• Our team’s goal is to build the foundations to democratize AI and Machine Learning for Atlassian’s teams, customers, and ecosystem. We aim to build productive, reliable tools that empower Atlassian teams to harness AI. These tools will facilitate the development, deployment, measurement, and operation of AI & ML models and experiences.

About This Role

As a Senior ML System Engineer on the AI & ML Platform’s Inference team, you will design and optimize large-scale model serving systems end-to-end. You will own everything from distributed infrastructure (global KV cache, continuous batching, load balancing, auto-scaling) to deep low-level optimizations (GPU kernels, quantization, speculative decoding).

In this role, you are expected to:

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems that serve billions of requests reliably
• Benchmark, fine-tune, and accelerate inference engines.
• Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.
• Partner with senior ML engineers to fine-tune open-source LLMs and deploy

On your first day, we’ll expect you to have

• 5+ years of software engineering experience, 2+ years of system performance optimization experience
• Deep low-level systems programming (C/C++ or Rust)
• Experience with large-scale, high-concurrent production serving.
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).
• Strong background in system optimizations: batching, caching, load balancing, parallelism.

It would be great, but not required if you have

• Low-level inference optimizations: GPU kernels
• Algorithmic inference optimizations: quantization, speculative decoding, distillation
• Experience with testing, benchmarking, and reliability of inference services.
• Experience designing and implementing CI/CD infrastructure for inference.

Compensation

This role may also be eligible for benefits, bonuses, commissions, and equity.

Pay Ranges

In The United States, we have three geographic pay zones. For this role, our current base pay ranges for new hires in each zone are:

Zone A: $180,000 - $235,000

Zone B: $162,000 - $211,500

Zone C: $149,400 - $195,050

Benefits & Perks

Atlassian offers a wide range of perks and benefits designed to support you, your family and to help you engage with your local community. Our offerings include health and wellbeing resources, paid volunteer days, and so much more.

In line with local law, identity verification (which may include use of biometric data) is a condition of employment with Atlassian for employment fraud purposes.