Senior ML Inference Systems Engineer - Remote-Friendly

Atlassian; Mountain View (CA); Hybrid; USD 206,000 - 269,000; Full time

Benefits offered by this job

Health & wellbeing

Volunteer days

Community perks

Job summary

At Atlassian, a Senior ML System Engineer on the AI & ML Platform’s Inference Team designs and optimizes large-scale model serving systems, covering distributed infrastructure, batching, and low-level GPU kernels.

You will work with senior ML engineers to fine‑tune open-source LLMs, implement CI/CD for inference, and push performance to meet billions of requests daily across multiple zones. Mountain View offers flexible work options with hybrid arrangements.

Qualifications

• 5+ years of software engineering experience, 2+ years of system performance optimization experience.
• Deep low-level systems programming (C/C++ or Rust).
• Experience with large-scale, high-concurrent production serving.

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems that serve billions of requests reliably
• Benchmark, fine-tune, and accelerate inference engines.
• Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.
• Partner with senior ML engineers to fine‑tune open-source LLMs and deploy

Skills

Software engineering

Performance optimization

Low-level programming

Large-scale serving

GPU inference engines

System optimizations

Job description

At Atlassian, a Senior ML System Engineer on the AI & ML Platform’s Inference Team designs and optimizes large-scale model serving systems, covering distributed infrastructure, batching, and low-level GPU kernels.

You will work with senior ML engineers to fine‑tune open-source LLMs, implement CI/CD for inference, and push performance to meet billions of requests daily across multiple zones. Mountain View offers flexible work options with hybrid arrangements.