LLM Inference Systems Architect

Netpreme; Santa Clara (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Relocation assistance

Visa sponsorship

Lunch stipend

401k match

Job summary

Netpreme is seeking an LLM Systems Engineer to prototype and optimize advanced inference systems on cutting-edge hardware. The role blends engineering with research, guiding hardware teams on product definitions and pushing the frontiers of ML inference software.

The candidate should have a strong track record in ML systems research and deep familiarity with industry-standard LLM inference systems, accelerator programming, and performance engineering.

Qualifications

• MS or PhD in computer systems, ideally with a focus on LLM inference and/or distributed systems.
• Prior experience contributing to the core LLM inference infrastructures (vLLM, SGLang, TensorRT, etc.).
• Prior experience in accelerator programming (e.g. CUDA, JAX/Pallas, ROCm).
• Advanced computer architectures and performance engineering skills.

Responsibilities

• Prototype and optimize emerging ML inference systems.
• Develop novel memory models for expandable vRAM.
• Write efficient GPU kernels for data movement.
• Perform design-space exploration, implementation, and benchmarking of inference engines, both in simulations and on real hardware.

Skills

LLM inference

GPU performance

Distributed systems

Performance engineering

Education

MS or PhD in computer systems

Tools

CUDA

JAX/Pallas

ROCm

TensorRT

Job description

Netpreme is seeking an LLM Systems Engineer to prototype and optimize advanced inference systems on cutting-edge hardware. The role blends engineering with research, guiding hardware teams on product definitions and pushing the frontiers of ML inference software.

The candidate should have a strong track record in ML systems research and deep familiarity with industry-standard LLM inference systems, accelerator programming, and performance engineering.