Senior Machine Learning Engineer

Cloudflare, Inc.; New York (NY); Hybrid; USD 194,000 - 266,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

401(k) Retirement Savings Plan

Employee Stock Participation Plan

Job summary

Cloudflare, Inc. is building production-grade ML inference to power experiences across its global network. This hybrid role in New York, NY focuses on low latency, reliability, and efficient resource use for models across GPUs and accelerators, from frontier LLMs to customer-deployed models.

You’ll drive benchmarking, validation, and deployment workflows while mentoring engineers. You will collaborate with product and engineering teams to translate customer needs into scalable ML capabilities

Qualifications

• Experience building ML models in production environments.
• Strong Python skills and ML framework proficiency (PyTorch, TensorFlow, JAX or equivalent).
• Hands-on inference optimization for large-scale models (quantization, batching, caching, compilation, runtime tuning).
• Experience with large-scale inference serving frameworks/runtimes (e.g., SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp).
• Familiarity with LLMs, speech/vision/embeddings/multimodal models and retrieval-augmented generation.
• Experience optimizing models for GPUs or specialized accelerators.
• Strong production ML foundations: evaluation, monitoring, regressions, rollout safety, reliability.
• Ability to work across ML and systems boundaries; distributed systems and serverless familiarity.
• Track record leading complex technical projects and mentoring engineers.

Responsibilities

• Develop, optimize, and productionize ML models for Cloudflare’s serverless inference platform.
• Create benchmarking and evaluation frameworks for latency, throughput, cost, and model behavior.
• Improve inference performance using quantization, batching, caching, compilation, runtime tuning, and accelerator awareness.
• Collaborate with systems engineers to integrate models into distributed inference infrastructure across GPUs and accelerators.
• Advance deployment workflows including validation, rollout safety, observability, and testing.
• Partner with product and engineering teams to translate customer requirements into scalable ML capabilities for Workers AI.
• Mentor engineers and contribute to technical direction to raise production ML quality.

Skills

Machine learning models

Python

ML frameworks

Inference optimization

Distributed systems

GPU acceleration

Model benchmarking

Mentoring engineers

Tools

Python

PyTorch

TensorFlow

JAX

SGLang

vLLM

TensorRT-LLM

ONNX Runtime

Triton

llama.cpp

Workers AI

Job description

Build production-grade machine learning inference that powers experiences across Cloudflare’s global network. In this hybrid role in New York, NY, you’ll help shape how models run from frontier open LLMs and real-time voice workloads to customer-deployed models served across heterogeneous GPUs and next-generation accelerators. You’ll focus on low latency, strong reliability, efficient resource use, and rigorous benchmarking and evaluation.

Cloudflare also supports your wellbeing and financial security with comprehensive coverage, time off, and retirement planning, plus eligibility for an Employee Stock Participation Plan and participation in the company equity program.

What you’ll do

• Develop, optimize, and productionize machine learning models for Cloudflare’s serverless inference platform, emphasizing performance, reliability, and model quality.
• Create benchmarking and evaluation frameworks to measure latency, throughput, cost efficiency, and model behavior across LLMs, speech, vision, and other model families.
• Improve inference performance using quantization, batching, caching, model compilation, runtime tuning, and accelerator-aware optimization.
• Work with systems engineers to integrate models into Cloudflare’s distributed inference infrastructure across a heterogeneous fleet of GPUs and next-generation accelerators.
• Advance model deployment workflows, including validation, rollout safety, observability, regression testing, and operational readiness.
• Collaborate with product and engineering teams to translate customer requirements into scalable ML capabilities for Workers AI.
• Mentor engineers, contribute to technical direction, and help raise the quality bar for production ML practices.

What you bring

• Experience building, optimizing, and operating machine learning models in production environments.
• Strong Python skills and modern ML framework proficiency including PyTorch, TensorFlow, JAX, or equivalent.
• Hands‑on inference optimization for large-scale models, including quantization, batching, caching, compilation, and serving runtime tuning.
• Experience with large-scale inference serving frameworks or runtimes such as SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp, or similar.
• Familiarity with LLMs, speech models, vision models, embeddings, multimodal models, retrieval‑augmented generation, or other modern deep learning architectures.
• Experience optimizing models for GPUs or specialized accelerators.
• Strong production ML foundations, including evaluation, monitoring, model regressions, rollout safety, and reliability.
• Ability to work across ML and systems boundaries, with familiarity with distributed systems, networking, or serverless platforms.
• Track record leading complex technical projects and mentoring other engineers.

Tools you may work with

Python, PyTorch, TensorFlow, JAX, SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp, Workers AI

Compensation

USD 194,000 - 266,000 per year (New York, NY hybrid). Compensation may be adjusted depending on work location. Bay Area hires have an estimated annual salary of $194,000 - $266,000, and New York City, New Jersey, Washington, Washington DC, and California (excluding Bay Area) hires have an estimated annual salary of $185,000 - $254,000.

Equity

This role is eligible to participate in Cloudflare’s equity plan.

Benefits

• Medical/Rx Insurance
• Dental Insurance
• Vision Insurance
• Flexible Spending Accounts
• Commuter Spending Accounts
• Fertility & Family Forming Benefits
• On‑demand mental health support and Employee Assistance Program
• Global Travel Medical Insurance
• Short and Long Term Disability Insurance
• Life & Accident Insurance
• 401(k) Retirement Savings Plan
• Employee Stock Participation Plan

Time off and leave

• Flexible paid time off covering vacation and sick leave
• Leave programs, including parental, pregnancy health, medical, and bereavement leave

Bonus points

• Experience contributing to open source ML tooling, model serving frameworks, or inference runtimes.