Senior ML Inference Engineer - Scalable Model Serving

CLOUDERA SINGAPORE PTE. LTD.; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

Cloudera is seeking a Staff ML Inference Engineer to develop and optimize production model-serving solutions through Cloudera AI Inference. You will work across on-premises, private-cloud, and public-cloud environments to translate customer requirements into reliable model endpoints.

The role focuses on inference and performance, benchmarking, bottleneck diagnosis, and reusable tooling, with strong collaboration across engineering teams to drive product improvements and deployment guidance.

Qualifications

• Eight or more years of engineering experience in ML systems, software development, distributed systems, or performance engineering with leadership.
• Experience operating GPU-based model-serving workloads in production using an inference stack (NVIDIA NIM, vLLM, SGLang, TensorRT-LLM, Triton or equivalent).
• Ability to turn ambiguous requirements into testable hypotheses and validate with experiments, profiling, and measurement.
• Strong Python programming and solid Linux/containers/Kubernetes foundations enabling issue investigation across the serving stack.
• Proven track record benchmarking and improving inference performance, understanding weights, runtime overhead, limitations, and latency-throughput trade-offs.
• Experience sizing GPU resources and serving configurations against workload demand including context lengths, concurrency, and bursts.

Responsibilities

• Deploy and optimize production model-serving solutions across on-premises, private-cloud, and public-cloud environments.
• Develop repeatable benchmarks for input/output lengths, request rates, and concurrency; recommend GPU types and configurations.
• Diagnose bottlenecks across runtimes, GPU memory, CPU processing, networking, and I/O; evaluate optimizations like batching and quantization.
• Improve resource utilization while balancing latency, throughput, cost, and constraints including GPU sharing/partitioning.
• Build benchmarking, profiling, and validation tooling to reproduce customer scenarios and measure latency and memory.
• Collaborate with customers, platform engineers, and core engineering to translate findings into product requirements and deployment guidance.

Skills

ML systems

Software development

Distributed systems

Performance engineering

Python programming

Linux

Containers

Kubernetes

Technical leadership

GPU model serving

Tools

NVIDIA NIM

vLLM

SGLang

TensorRT-LLM

Triton

Job description

Cloudera is seeking a Staff ML Inference Engineer to develop and optimize production model-serving solutions through Cloudera AI Inference. You will work across on-premises, private-cloud, and public-cloud environments to translate customer requirements into reliable model endpoints.

The role focuses on inference and performance, benchmarking, bottleneck diagnosis, and reusable tooling, with strong collaboration across engineering teams to drive product improvements and deployment guidance.