Staff Software Engineer - GenAI inference

FDE Times; San Francisco (CA); Presencial; USD 180.000 - 280.000; Jornada completa

Descripción de la vacante

FDE Times in San Francisco seeks a senior architect to lead the design and optimization of a GenAI inference engine and model-serving stack. You will drive end-to-end architecture, routing, batching, scheduling, instrumentation, and distributed orchestration with researchers and platform teams.

The role requires advanced degrees in CS and extensive hands-on CUDA/GPU programming, distributed systems expertise, and a proven record of performance optimization in latency-sensitive software.

Formación

• Requires a degree in CS or related field and strong end-to-end architecture ownership.
• Deep ML inference knowledge with CUDA and GPU programming experience.
• Experience with profiling and performance optimization in distributed systems.

Responsabilidades

• Lead architecture, development, and optimization of GenAI inference engine.
• Improve latency, throughput, memory efficiency, and hardware utilization.
• Build scalable inference capabilities including routing, batching, scheduling, instrumentation, and orchestration.

Conocimientos

GenAI Inference

LLM Model Serving

CUDA

GPU Programming

Distributed Systems

Performance Optimization

Inference Engine Architecture

Machine Learning Inference

Quantization

Sparse Operations

Profiling and Tracing

Memory Management

Scheduling and Batching

Fault Tolerance

Cross-Functional Leadership

Educación

Bachelor’s/Master’s/PhD in CS or related field

Descripción del empleo

Lead the architecture, development, and optimization of Databricks’ GenAI inference engine and model-serving stack, improving latency, throughput, memory efficiency, and hardware utilization. Build scalable and reliable inference capabilities—including routing, batching, scheduling, instrumentation, and distributed orchestration—in collaboration with researchers and platform teams.

Requirements

Requires a bachelor’s, master’s, or doctoral degree in computer science or a related field, along with at least six years of experience in performance-critical software systems and a record of owning complex architecture end to end. Candidates should have deep ML inference knowledge, hands-on CUDA and GPU programming experience, distributed systems expertise, and experience with profiling and performance optimization.

Key Skills

Software Engineering, GenAI Inference, LLM Model Serving, CUDA, GPU Programming, Distributed Systems, Performance Optimization, Inference Engine Architecture, Machine Learning Inference, Quantization, Sparse Operations, Profiling and Tracing, Memory Management, Scheduling and Batching, Fault Tolerance, Cross-Functional Leadership