MTS Inference: GPU Kernel & Performance Architect

Sail Research; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Meals provided

Studio Display for every employee

Job summary

Sail Research in San Francisco is looking for a motivated software engineer to optimize token processing at every layer of the stack. You will modify inference engines and analyze GPU performance, ensuring efficient hardware utilization.

The ideal candidate has a solid understanding of LLM mechanics and interests in cutting-edge MLSys research. The company offers benefits like free meals and a Studio Display for every employee.

Qualifications

• Strong understanding of LLM mechanics, such as KV cache and mixture-of-experts.
• Interest in MLSys research and current advancements.
• Familiarity or willingness to learn modern tile-based GPU programming.

Responsibilities

• Modify and extend state-of-the-art inference engines.
• Analyze GPU time utilization and explain kernel launches.
• Design and implement parallelism schemes for unique hardware.

Skills

Understanding of LLM mechanics

GPU programming

Interest in MLSys research

Tools

Triton

CUTLASS

ThunderKittens

Job description

Sail Research in San Francisco is looking for a motivated software engineer to optimize token processing at every layer of the stack. You will modify inference engines and analyze GPU performance, ensuring efficient hardware utilization.

The ideal candidate has a solid understanding of LLM mechanics and interests in cutting-edge MLSys research. The company offers benefits like free meals and a Studio Display for every employee.