Machine Learning System Engineer

Atlassian Corp.; Seattle, Northern (WA, KY); Hybrid; USD 178,000 - 233,000; Full time

Benefits offered by this job

Health and wellbeing resources

Volunteer days

Job summary

At Atlassian, the ML System Engineer will design and optimize large-scale model serving systems, spanning distributed infrastructure to low-level GPU kernel optimizations. You’ll own end-to-end components from caching and batching to auto-scaling and deployment.

The role emphasizes building reliable, high-concurrency serving systems, benchmarking and tuning inference engines, and partnering with senior ML engineers to deploy open-source LLMs.

Qualifications

• 3+ years of software engineering experience.
• Deep low-level systems programming in C/C++ or Rust.
• Experience with large-scale, high-concurrent production serving.
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).

Responsibilities

• Architect scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling).
• Optimize latency and throughput under production workloads.
• Build high-concurrency serving systems that handle billions of requests.
• Benchmark, fine-tune, and accelerate inference engines.

Skills

System programming

Distributed systems

GPU inference engines

High-concurrency serving

C/C++ or Rust

Tools

GPU frameworks

Job description

Engineering | Seattle, United States | Remote, Remote | San Francisco, United States |

Atlassians can choose where they work – whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity.

As a ML System Engineer on the AI & ML Platform’s Inference team, you will design and optimize large-scale model serving systems end-to-end. You will have the chance to own everything from distributed infrastructure (global KV cache, continuous batching, load balancing, auto-scaling) to deep low-level optimizations (GPU kernels, quantization, speculative decoding).

In this role, you are expected to:

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems that serve billions of requests reliably
• Benchmark, fine-tune, and accelerate inference engines.
• Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.
• Partner with senior ML engineers to fine‑tune and deploy open‑source LLMs

At Atlassian, we strive to design equitable, explainable, and competitive compensation programs. We follow consistent hiring practices and account for each candidate's skills, knowledge, and experience when setting base pay within the range.

Please visit go.atlassian.com/payzones for more information on which locations are included in each of our geographic pay zones. However, please confirm the zone for your specific location with your recruiter.

This role may also be eligible for benefits, bonuses, commissions, and equity.

In The United States, we have three geographic pay zones. For this role, our current base pay ranges for new hires in each zone are:

Zone A: $178,200 - $232,650

Zone B: $160,380 - $209,385

Zone C: $147,906 - $193,100

On your first day, we’ll expect you to have:

• 3+ years of software engineering experience
• Deep low-level systems programming (C/C++ or Rust)
• Experience with large-scale, high-concurrent production serving.
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).

It would be great, but not required if you have:

• 1+ years of system performance optimization experience
• Low-level inference optimizations: GPU kernels
• Algorithmic inference optimizations: quantization, speculative decoding, distillation
• Experience with testing, benchmarking, and reliability of inference services.
• Experience designing and implementing CI/CD infrastructure for inference.
• Strong background in system optimizations: batching, caching, load balancing, parallelism.

Atlassian offers a wide range of perks and benefits designed to support you, your family and to help you engage with your local community. Our offerings include health and wellbeing resources, paid volunteer days, and so much more. To learn more, visit go.atlassian.com/perksandbenefits.

About Atlassian

At Atlassian, we're motivated by a common goal: to unleash the potential of every team. Our software products help teams all over the planet and our solutions are designed for all types of work. Team collaboration through our tools makes what may be impossible alone, possible together.

We believe that the unique contributions of all Atlassians create our success. To ensure that our products and culture continue to incorporate everyone's perspectives and experience, we never discriminate based on race, religion, national origin, gender identity or expression, sexual orientation, age, or marital, veteran, or disability status. All your information will be kept confidential according to EEO guidelines.

To provide you the best experience, we can support with accommodations or adjustments at any stage of the recruitment process. Simply inform our Recruitment team during your conversation with them.

In line with local law, identity verification (which may include use of biometric data) is a condition of employment with Atlassian for employment fraud purposes.