GenAI Inference Architect – High-Performance ML Serving

FDE Times; San Francisco (CA); On-site; USD 180,000 - 280,000; Full time

Job summary

FDE Times in San Francisco seeks a senior architect to lead the design and optimization of a GenAI inference engine and model-serving stack. You will drive end-to-end architecture, routing, batching, scheduling, instrumentation, and distributed orchestration with researchers and platform teams.

The role requires advanced degrees in CS and extensive hands-on CUDA/GPU programming, distributed systems expertise, and a proven record of performance optimization in latency-sensitive software.

Qualifications

• Requires a degree in CS or related field and strong end-to-end architecture ownership.
• Deep ML inference knowledge with CUDA and GPU programming experience.
• Experience with profiling and performance optimization in distributed systems.

Responsibilities

• Lead architecture, development, and optimization of GenAI inference engine.
• Improve latency, throughput, memory efficiency, and hardware utilization.
• Build scalable inference capabilities including routing, batching, scheduling, instrumentation, and orchestration.

Skills

GenAI Inference

LLM Model Serving

CUDA

GPU Programming

Distributed Systems

Performance Optimization

Inference Engine Architecture

Machine Learning Inference

Quantization

Sparse Operations

Profiling and Tracing

Memory Management

Scheduling and Batching

Fault Tolerance

Cross-Functional Leadership

Education

Bachelor’s/Master’s/PhD in CS or related field

Job description

FDE Times in San Francisco seeks a senior architect to lead the design and optimization of a GenAI inference engine and model-serving stack. You will drive end-to-end architecture, routing, batching, scheduling, instrumentation, and distributed orchestration with researchers and platform teams.

The role requires advanced degrees in CS and extensive hands-on CUDA/GPU programming, distributed systems expertise, and a proven record of performance optimization in latency-sensitive software.