Staff Backend Software Engineer- (AI Platform)

FDE Times; San Francisco (CA); On-site; USD 190,000 - 260,000; Full time

Job summary

FDE Times is seeking a senior leader to design and operate scalable model serving systems and APIs. The role focuses on optimizing inference performance across CPU and GPU workloads and shaping the technical roadmap.

You will mentor engineers, drive latency and cost improvements, and collaborate across product, platform, infrastructure, and research teams to deliver high-impact initiatives.

Qualifications

• 10+ years building and operating large-scale distributed systems.
• Deep expertise in model serving and inference infrastructure.
• Strong system design and algorithmic foundations.
• Proven track record delivering high-impact technical initiatives.
• Excellent communication and mentoring skills.

Responsibilities

• Design scalable model serving systems and APIs to optimize latency and throughput.
• Lead initiatives to improve latency, availability, and cost across CPU/GPU workloads.
• Shape roadmap and architecture and mentor engineers.
• Collaborate with product, platform, infrastructure, and research teams.

Skills

Distributed Systems

Model Serving

Inference Systems

System Design

Algorithms

Data Structures

CPU/GPU Workloads

Routing

Scheduling

Autoscaling

Observability

Performance

Technical Leadership

Cross-Functional

Mentoring

Job description

Design and build scalable, reliable Model Serving systems and APIs, optimizing inference performance, throughput, autoscaling, and operational efficiency across CPU and GPU workloads. Shape the technical roadmap and architecture, lead initiatives to improve latency, availability, and cost, and mentor engineers while collaborating across product, platform, infrastructure, and research teams.

Requirements: Requires 10+ years of experience building and operating large-scale distributed systems, with deep expertise in model serving, inference infrastructure, and performance-sensitive CPU/GPU systems. Candidates should demonstrate strong system design and algorithmic foundations, a record of delivering high-impact technical initiatives, strategic product thinking, communication skills, and a commitment to mentoring.

Key Skills: Distributed Systems, Model Serving, Inference Systems, System Design, Algorithms, Data Structures, CPU/GPU Workloads, Routing, Scheduling, Autoscaling, Observability, Performance Optimization, Technical Leadership, Cross-Functional Collaboration, Mentoring