LLM Inference Architect & Systems Optimizer

Togetherai; San Francisco (CA); On-site; USD 160,000 - 230,000; Full time

Benefits offered by this job

Health insurance

Startup equity

Competitive benefits

Job summary

Togetherai in San Francisco seeks an Inference Frameworks and Optimization Engineer to design scalable inference engines for large language models. As part of a dynamic team, you will work on optimizing performance and collaborating closely with hardware teams.

The role requires a strong background in deep learning frameworks and programming with Python and C++. Competitive compensation includes a salary between $160,000 - $230,000 plus equity.

Qualifications

• 3+ years of experience in deep learning inference frameworks.
• Familiarity with LLM inference frameworks.
• Proficient in Python and C++/CUDA.

Responsibilities

• Design and develop high-concurrency distributed inference engines.
• Collaborate with hardware teams to optimize performance.
• Implement distributed inference strategies for performance.

Skills

Deep learning inference frameworks

Distributed systems

High-performance computing

Python

C++/CUDA

Transformer architecture optimization

Tools

TensorRT

CUDA

Kubernetes

Job description

Togetherai in San Francisco seeks an Inference Frameworks and Optimization Engineer to design scalable inference engines for large language models. As part of a dynamic team, you will work on optimizing performance and collaborating closely with hardware teams.

The role requires a strong background in deep learning frameworks and programming with Python and C++. Competitive compensation includes a salary between $160,000 - $230,000 plus equity.