ML Inference Engineer - High-Performance AI Systems

Together Computer Inc; San Francisco (CA); On-site; USD 200,000 - 300,000; Full time

Benefits offered by this job

Startup equity

Health insurance

Competitive compensation

Job summary

Together AI is seeking a Machine Learning Engineer to strengthen our Inference Engine team. You will design production systems powering AI inference at scale, optimize runtime services for large-scale AI applications, and collaborate with researchers, engineers, and product teams to bring cutting-edge features to market.

The role requires 3+ years of production-quality coding experience, proficiency in Python and PyTorch, and a strong grasp of low-level OS concepts, multithreading, memory

Qualifications

• 3+ years of experience writing production-quality code.
• Proficiency with Python and PyTorch.
• Experience building high-performance libraries and tooling.
• Strong understanding of low-level OS concepts including multi-threading, memory management, networking, storage, performance, and scale.

Responsibilities

• Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale.
• Develop and optimize runtime inference services for large-scale AI applications.
• Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world.
• Conduct design and code reviews to ensure high standards of quality.
• Create services, tools, and developer documentation to support the inference engine.
• Implement robust and fault-tolerant systems for data ingestion and processing.

Skills

Python

PyTorch

High-performance libraries

Multi-threading

CUDA/Triton

Tools

TGI

vLLM

TensorRT-LLM

Optimum

Rust

Cython

Compilers

Job description

Together AI is seeking a Machine Learning Engineer to strengthen our Inference Engine team. You will design production systems powering AI inference at scale, optimize runtime services for large-scale AI applications, and collaborate with researchers, engineers, and product teams to bring cutting-edge features to market.

The role requires 3+ years of production-quality coding experience, proficiency in Python and PyTorch, and a strong grasp of low-level OS concepts, multithreading, memory