Senior ML Engineer, Production Inference & Optimization

Cloudflare; New York (NY); Hybrid; USD 185,000 - 254,000; Full time

Benefits offered by this job

Health Insurance

401(k) Retirement Savings Plan

Employee Stock Participation Plan

Flexible Paid Time Off

Parental & family leave

Job summary

Cloudflare is seeking a machine learning engineer to define how models run across its global network. You will work with ML and systems teams to bring frontier open LLMs and real-time models into production with low latency and high reliability.

You will optimize inference, build evaluation frameworks, and collaborate on deployment tooling for Workers AI. The role emphasizes mentoring and shaping production ML practices within a large-scale, AI-driven environment.

Qualifications

• Production ML experience with model serving in large-scale systems.
• Proficiency with Python and ML frameworks (PyTorch, TensorFlow, JAX or equivalent).
• Hands-on inference optimization techniques (quantization, batching, caching, compilation).
• Experience with large-scale inference serving frameworks/runtimes (SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp).
• Familiarity with LLMs, speech/vision models and multimodal architectures.
• Experience optimizing models for GPUs or accelerators.
• Strong production ML concerns: evaluation, monitoring, rollout safety, reliability.

Responsibilities

• Develop, optimize, and productionize ML models on Cloudflare's serverless inference platform.
• Build benchmarking frameworks to measure latency, throughput, and cost across model families.
• Improve inference performance via quantization, batching, caching, and runtime tuning.
• Collaborate to integrate models into Cloudflare’s distributed inference infrastructure across GPUs and accelerators.
• Drive improvements to deployment workflows, validation, rollout safety, and observability.
• Partner with product and engineering teams to translate customer needs into scalable ML capabilities.
• Mentor engineers and contribute to technical direction and best practices.

Skills

Python

ML frameworks

Inference optimization

Distributed systems

Benchmarking

Mentoring

Open source contributions

Tools

PyTorch

TensorFlow

JAX

SGLang

vLLM

TensorRT-LLM

ONNX Runtime

Triton

llama.cpp

Job description

Cloudflare is seeking a machine learning engineer to define how models run across its global network. You will work with ML and systems teams to bring frontier open LLMs and real-time models into production with low latency and high reliability.

You will optimize inference, build evaluation frameworks, and collaborate on deployment tooling for Workers AI. The role emphasizes mentoring and shaping production ML practices within a large-scale, AI-driven environment.