Member of Technical Staff (AI Inference Engineer)

Kindredventures; Palo Alto (CA); Hybrid; USD 190,000 - 250,000; Full time

Benefits offered by this job

Comprehensive health insurance

Dental insurance

Vision insurance

401(k) plan

Job summary

A leading financial technology firm in California seeks an AI Inference engineer to join its team. The role involves developing APIs for AI inference, improving system reliability, and optimizing LLM performance. Required qualifications include experience with ML systems, deep learning frameworks, and GPU programming. The position offers a hybrid work model and competitive salary ranging from $190,000 to $250,000, alongside health benefits and a 401(k) plan.

Qualifications

• Experience with ML systems and deep learning frameworks required.
• Familiarity with LLM architectures and optimization techniques necessary.
• Understanding of GPU architectures and CUDA programming.

Responsibilities

• Develop APIs for AI inference for internal and external users.
• Benchmark and resolve bottlenecks in inference stack.
• Enhance system reliability and respond to outages.
• Research and implement LLM inference optimizations.

Skills

Experience with ML systems

Deep learning frameworks (e.g. PyTorch, TensorFlow)

Common LLM architectures knowledge

Inference optimization techniques

Understanding of GPU architectures

GPU kernel programming using CUDA

Tools

Python

Rust

C++

PyTorch

Triton

CUDA

Kubernetes

Job description

Location

San Francisco

Employment Type

Full time

Location Type

Hybrid

Department

AI

We are looking for an AI Inference engineer to join our growing team. Our current stack is Python, Rust, C++, PyTorch, Triton, CUDA, Kubernetes. You will have the opportunity to work on large-scale deployment of machine learning models for real-time inference.

Responsibilities

• Develop APIs for AI inference that will be used by both internal and external customers

• Benchmark and address bottlenecks throughout our inference stack

• Improve the reliability and observability of our systems and respond to system outages

• Explore novel research and implement LLM inference optimizations

Qualifications

• Experience with ML systems and deep learning frameworks (e.g. PyTorch, TensorFlow, ONNX)

• Familiarity with common LLM architectures and inference optimization techniques (e.g. continuous batching, quantization, etc.)

• Understanding of GPU architectures or experience with GPU kernel programming using CUDA

The cash compensation range for this role is $190,000 - $250,000.

Final offer amounts are determined by multiple factors, including, experience and expertise, and may vary from the amounts listed above.

Equity: In addition to the base salary, equity may be part of the total compensation package. Benefits: Comprehensive health, dental, and vision insurance for you and your dependents. Includes a 401(k) plan.