Senior ML Inference Engineer — Production & Systems

FDE Times; Aceh; Hybrid; IDR 3,349,022,000 - 4,598,117,000; Full time

Benefits offered by this job

Equity plan

Medical Insurance

401(k) Retirement Savings Plan

Employee Stock Participation Plan

Flexible paid time off

Job summary

Cloudflare is seeking an ML/AI systems engineer to define how machine learning models run across its global network, from frontier open LLMs to customer-deployed models, with low latency, high reliability, and efficient resource use. You’ll work with systems engineers, product teams, hardware partners, and AI/ML engineers to bring models into production.

You’ll focus on benchmarking, evaluation, and tooling to ship AI applications at Internet scale, collaborating to improve serving performance

Qualifications

• Experience building, optimizing, and operating machine learning models in production environments.
• Strong proficiency with Python and modern ML frameworks such as PyTorch, TensorFlow, JAX, or equivalent.
• Hands-on experience with inference optimization techniques for large-scale models, including quantization, batching, caching, compilation, and serving runtime tuning.

Responsibilities

• Develop, optimize, and productionize machine learning models for Cloudflare’s serverless inference platform, with a focus on performance, reliability, and model quality.
• Build benchmarking and evaluation frameworks to measure latency, throughput, cost efficiency, and model behavior across LLMs, speech, vision, and other model families.
• Improve inference performance through quantization, batching, caching, model compilation, runtime tuning, and accelerator-aware optimization.
• Partner with systems engineers to integrate models into Cloudflare’s distributed inference infrastructure across a heterogeneous fleet of GPUs and next-generation accelerators.
• Drive improvements to model deployment workflows, including validation, rollout safety, observability, regression testing, and operational readiness.
• Collaborate with product and engineering teams to translate customer requirements into scalable ML capabilities for Workers AI.
• Mentor engineers, contribute to technical direction, and raise the quality bar for production ML engineering practices across the team.

Skills

Python

PyTorch

TensorFlow

JAX

Inference optimization

LLMs

Production ML

Distributed systems

GPU acceleration

Model deployment

Observability

Rollout safety

Tools

SGLang

vLLM

TensorRT-LLM

ONNX Runtime

Triton

llama.cpp

Job description

Cloudflare is seeking an ML/AI systems engineer to define how machine learning models run across its global network, from frontier open LLMs to customer-deployed models, with low latency, high reliability, and efficient resource use. You’ll work with systems engineers, product teams, hardware partners, and AI/ML engineers to bring models into production.

You’ll focus on benchmarking, evaluation, and tooling to ship AI applications at Internet scale, collaborating to improve serving performance