Staff Software Engineer- Foundation Model Inference

FDE Times; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

FDE Times in San Francisco is seeking backend and infrastructure engineers to build and operate the infrastructure for large-scale LLM inference across partner and self-hosted models.

You will collaborate with platform, infrastructure, and machine learning teams to deliver end-to-end AI experiences and shape how developers and data scientists use AI on Databricks.

Qualifications

• Eight years or more of backend or infrastructure engineering experience.
• Experience with distributed systems, scalable APIs, or cloud-native infrastructure.
• Sought experience in real-time serving, ML infrastructure, or GPU orchestration.

Responsibilities

• Build and operate infrastructure for large-scale LLM inference across partner and self-hosted models.
• Collaborate with platform, infrastructure, and ML teams to deliver end-to-end AI experiences and shape how developers and data scientists use AI on Databricks.

Skills

Backend Engineering

Infrastructure Engineering

Distributed Systems

Scalable APIs

Cloud-Native Infrastructure

Real-Time Serving

Machine Learning Infrastructure

GPU Orchestration

Service-Oriented Architecture

Deployment Pipelines

System Observability

LLM Inference

AI Infrastructure

Developer Platforms

Job description

Build and operate infrastructure for large-scale LLM inference across partner and self-hosted models, improving the reliability, latency, and efficiency of distributed AI workloads. Collaborate with platform, infrastructure, and machine learning teams to deliver end-to-end AI experiences and shape how developers and data scientists use AI on Databricks.

Requirements: Candidates should have at least eight years of backend or infrastructure engineering experience and experience with distributed systems, scalable APIs, or cloud-native infrastructure. Experience with real-time serving, machine learning infrastructure, or GPU orchestration is sought; familiarity with service-oriented architecture, deployment pipelines, and observability is also relevant.

Key Skills: Backend Engineering, Infrastructure Engineering, Distributed Systems, Scalable APIs, Cloud-Native Infrastructure, Real-Time Serving, Machine Learning Infrastructure, GPU Orchestration, Service-Oriented Architecture, Deployment Pipelines, System Observability, LLM Inference, AI Infrastructure, Developer Platforms