AI Inference Kernel Optimization Engineer Remote · Bangalore, India

Vidman

Bengaluru

Remote

INR 4,000,000 - 7,000,000

Full time

9 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Vidman AI is seeking an experienced AI Inference Kernel Optimization Engineer to join a remote-first team. You will craft and tune compute kernels, optimise memory paths, and implement low-precision inference across GPUs to reduce latency and cost.

You will profile at the hardware level and work with the serving runtimes to push improvements into production. A strong background in CUDA, Python, and low-level GPU work is essential.

Qualifications

  • 5+ years in systems software or ML systems engineering with real time GPU performance work.
  • Deep proficiency in C++ and Python, plus at least one low-level parallel framework (CUDA, Triton or CUTLASS).
  • Experience reading assembly-level code (PTX/SASS) and profiling hardware efficiently.
  • Familiarity with modern serving stacks (vLLM, TensorRT-LLM) and transformer inference behavior.

Responsibilities

  • Write and tune custom compute kernels (attention, matmul, normalisation) in CUDA, Triton, and CUTLASS.
  • Optimise memory movement and KV-cache management for high throughput.
  • Implement low-precision inference (INT4, FP8, INT8) aligned with hardware.
  • Profile with Nsight Systems, Compute Sanitizer, and PTX/SASS for shipped improvements.
  • Work within deployed runtimes (vLLM, TensorRT-LLM) and upstream relevant changes.

Skills

C++
Python
CUDA
Triton
CUTLASS
GPU profiling
PTX/SASS

Education

Bachelor's degree in CS/CE/EE

Tools

Nsight Systems
Compute Sanitizer
PTX/SASS
TensorRT-LLM
vLLM

Job description

# AI Inference Kernel Optimization EngineerEngineering5+ yearsRemote · Bangalore, IndiaVidman AI is a capacity business behind an API: every fraction of utilization won in the serving stack is room to cut the price of a token. As an AI Inference Kernel Optimization Engineer you work at the layer where that is decided: the compute kernels, memory paths, and serving runtimes that execute open models on our GPUs. Your work shows up in two places customers feel: latency, and the price list.## What you'll do* 01Write and tune custom compute kernels (attention, matrix multiplication, normalisation) in CUDA, Triton, and CUTLASS where the serving stack leaves performance on the table.* 02Optimise memory movement (cache reuse, KV-cache management, operator fusion) so kernels are bound by math, not by bandwidth.* 03Implement low-precision inference (INT4, FP8, INT8 quantization paths) matched to the hardware actually run.* 04Profile at the hardware level with Nsight Systems, Compute Sanitizer, and PTX/SASS inspection, and act on what you find into shipped improvements.* 05Work inside the deployed serving runtimes (vLLM, TensorRT-LLM) and upstream what belongs upstream.* 06Partner with research and platform engineering so new model architectures land on kernels that can actually serve them.## What you bring* At least 5 years in systems software or ML systems engineering, with real time on GPU performance work.* Deep proficiency in C++ and Python, plus mastery of at least one low-level parallel programming framework: CUDA, Triton, or CUTLASS.* Working fluency with hardware-level profiling tools and the ability to read assembly-level execution (PTX/SASS) when the profiler is not enough.* Familiarity with the modern serving stack (vLLM, TensorRT-LLM, or a comparable runtime) and with how transformer inference actually spends its time and memory.* Evidence over adjectives: kernels written or tuned, with before-and-after numbers.## Nice to have* Experience with deep-learning compiler stacks: MLIR, LLVM, or torch.compile internals.* Contributions to vLLM, TensorRT-LLM, Triton, or another inference-relevant open-source project, readable by us.* KV-cache and paged-attention internals, past the tutorial level.* Hardware-software co-design exposure: working with compiler or silicon teams on next-generation hardware should support.* An advanced degree in CS, CE, or EE is welcome; equivalent depth built in production counts the same.## Apply for this roleFive minutes: your resume, your links, and a paragraph about why this one. We read every application.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Applied Machine Learning Engineer Remote · Bangalore, India
Applied Machine Learning Engineer Remote · Bangalore, India

Vidman • Bengaluru

Remote
INR 4,500,000 - 6,000,000
Senior System Software Engineer - Local AI
Senior System Software Engineer - Local AI

We Love AI Jobs • Pune District

On-site
INR 3,500,000 - 6,000,000
Performance Engineer, Inference
Performance Engineer, Inference

Sarvam • Bengaluru

Hybrid
INR 4,200,000 - 6,300,000
Research Engineer Carry routing, fine-tuning, and alignment research into production behaviour. Remote · Bangalore, India
Research Engineer Carry routing, fine-tuning, and alignment research into production behaviour. Remote · Bangalore, India

Vidman • Bengaluru

Remote
INR 2,500,000 - 3,500,000
Senior/Staff Deep Learning Compiler Engineer
Senior/Staff Deep Learning Compiler Engineer

We Love AI Jobs • Pune District

On-site
INR 4,000,000 - 7,000,000
Performance Engineer, Inference
Performance Engineer, Inference

Sarvam • Chennai District

Hybrid
INR 4,000,000 - 7,000,000
Hybrid work model
Performance Engineer, Kernels
Performance Engineer, Kernels

Sarvam • Bengaluru

Hybrid
INR 3,500,000 - 6,500,000
AI Product Engineer Remote · Bangalore, India
AI Product Engineer Remote · Bangalore, India

Vidman • Bengaluru

Remote
INR 4,200,000 - 6,600,000
AI Software Engineer - AI Frameworks
AI Software Engineer - AI Frameworks

Intel Technology India Pvt Ltd • Bengaluru

On-site
INR 4,000,000 - 7,000,000
Performance Engineer, Kernels
Performance Engineer, Kernels

Sarvam • Chennai District

Hybrid
INR 3,800,000 - 6,200,000