Staff Software Engineer- Foundation Model Inference

DevHub; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Annual performance bonus

Equity

Job summary

DevHub is building scalable LLM infrastructure to power large-scale inference workloads. You will work with cross-functional teams to improve reliability, latency, and efficiency of distributed AI systems in a fast-growing environment.

We seek a senior backend/infrastructure engineer with 8+ years' experience in distributed systems, scalable APIs, and cloud-native infrastructure. Expertise in ML infrastructure, GPU orchestration, and SOA is essential; PyTorch and vLLM experience is a plus.

Qualifications

• Requires 8+ years of experience in backend or infrastructure engineering with expertise in distributed systems and scalable APIs.
• Experience with ML infrastructure, GPU orchestration, and service-oriented architecture is essential.

Skills

Backend Engineering

Infrastructure Engineering

Distributed Systems

Scalable APIs

Cloud-native Infrastructure

Real-time Serving

ML Infrastructure

GPU Orchestration

Service-oriented Architecture

Deployment Pipelines

System Observability

LLM Infrastructure

Model Inference

Tools

PyTorch

vLLM

Job description

Build and optimize LLM infrastructure to power large-scale inference workloads for both partner and self-hosted models. Collaborate with cross-functional teams to improve the reliability, latency, and efficiency of distributed AI workloads.

Requirements:

• Requires over 8 years of experience in backend or infrastructure engineering with expertise in distributed systems and scalable APIs.
• Experience with ML infrastructure, GPU orchestration, and service-oriented architecture is essential.

Key Skills:

• Backend Engineering
• Infrastructure Engineering
• Distributed Systems
• Scalable APIs
• Cloud-native Infrastructure
• Real-time Serving
• ML Infrastructure
• GPU Orchestration
• Service-oriented Architecture
• Deployment Pipelines
• System Observability
• LLM Infrastructure
• Model Inference
• PyTorch
• vLLM

Benefits:

• Annual performance bonus
• Equity