Inference Runtime Engineer for LLMs & Diffusion

Inferact; San Francisco (CA); Hybrid; USD 200,000 - 400,000; Full time

Benefits offered by this job

Health, dental, and vision benefits

401(k) company match

Visa sponsorship on case-by-case basis

Job summary

Inferact is seeking an inference runtime engineer to enhance the performance and capabilities of LLM and diffusion model serving. This role requires expertise in optimizing model execution on various hardware architectures and has significant implications for AI inference.

The ideal candidate must possess a bachelor's degree in computer science or related fields, strong programming skills in Python, and experience with LLM inference systems. Remote work options are available for exceptional candidates.

Qualifications

• Bachelor's degree or equivalent experience in computer science, engineering, or similar.
• Deep understanding of transformer architectures and their variants.
• Strong programming skills in Python with experience in PyTorch internals.
• Experience with LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI).
• Ability to read and implement model architectures from research papers.

Responsibilities

• Push the boundaries of inference model serving.
• Optimize how models execute across diverse hardware.
• Directly impact how the world runs AI inference.

Skills

Programming skills in Python

Deep understanding of transformer architectures

Experience with LLM inference systems

Ability to read model architectures

Education

Bachelor's degree or equivalent experience

Tools

PyTorch

TensorRT-LLM

vLLM

Job description

Inferact is seeking an inference runtime engineer to enhance the performance and capabilities of LLM and diffusion model serving. This role requires expertise in optimizing model execution on various hardware architectures and has significant implications for AI inference.

The ideal candidate must possess a bachelor's degree in computer science or related fields, strong programming skills in Python, and experience with LLM inference systems. Remote work options are available for exceptional candidates.