Performance Engineer (Inference)

HCL Technologies Limited; Bengaluru; Presencial; INR 3.000.000 - 6.000.000; Jornada completa

Descripción de la vacante

HCL Technologies Limited in Bengaluru seeks an Inference Runtime Engineer to build and optimize the model-serving infrastructure that keeps GPU utilization high and cost per token low, across rollout generation and production serving.

You will tackle KV-cache management, batching, quantization, and intelligent routing, and collaborate with product engineering to expose capabilities via customer-facing APIs, reducing latency in internal and client workloads.

Formación

• 5+ years of experience in systems and ML infrastructure engineering.
• Strong experience with model-serving systems (e.g., vLLM, SGLang, TensorRT-LLM or comparable).
• Hands-on CUDA or Triton experience.
• Solid understanding of quantization and batching strategies for inference.
• Proficiency in C++ and Python.

Responsabilidades

• Build and optimize model-serving infrastructure: KV-cache management, continuous batching, and speculative decoding.
• Implement quantization (FP8/INT4) and intelligent routing to reduce serving cost per token.
• Support low-latency rollout serving for internal model training and customer workloads.
• Diagnose and resolve GPU utilization bottlenecks in production serving.
• Work with product engineering to expose serving capabilities through customer-facing APIs.

Conocimientos

5+ years in systems or ML infra

Model-serving systems

CUDA or Triton

Quantization and batching

C++ and Python

Educación

B.Tech in Computer Science or related field

M.Tech a plus for senior scope

Descripción del empleo

Serving models efficiently is its own discipline, distinct from training them. As an Inference Runtime Engineer, you'll build the serving infrastructure that keeps GPU utilization high and cost per token low across everything we run, from rollout generation to production model serving.

What You'll Do

• Build and optimize model-serving infrastructure: KV-cache management, continuous batching, and speculative decoding
• Implement quantization (FP8/INT4) and intelligent routing to reduce serving cost per token
• Support low-latency rollout serving for both internal model training and customer-facing workloads
• Diagnose and resolve GPU utilization bottlenecks in production serving
• Work with product engineering to expose serving capabilities through customer-facing APIs

What You'll Bring

• 5+ years of experience in systems or ML infrastructure engineering
• Strong experience with model-serving systems (vLLM, SGLang, TensorRT-LLM, or comparable)
• Hands-on CUDA or Triton experience
• Solid understanding of quantization and batching strategies for inference
• Proficiency in C++ and Python

Education

B.Tech in Computer Science or a related field; M.Tech a plus for senior scope.

Nice to Have

• Experience at an inference-focused AI infrastructure company or a hyperscaler's model-serving team
• Exposure to speculative decoding or other latency-reduction techniques in production