Staff Software Engineer: LLM Inference & GPU Infra (Equity)

DevHub; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Annual performance bonus

Equity

Job summary

DevHub is building scalable LLM infrastructure to power large-scale inference workloads. You will work with cross-functional teams to improve reliability, latency, and efficiency of distributed AI systems in a fast-growing environment.

We seek a senior backend/infrastructure engineer with 8+ years' experience in distributed systems, scalable APIs, and cloud-native infrastructure. Expertise in ML infrastructure, GPU orchestration, and SOA is essential; PyTorch and vLLM experience is a plus.

Qualifications

• Requires 8+ years of experience in backend or infrastructure engineering with expertise in distributed systems and scalable APIs.
• Experience with ML infrastructure, GPU orchestration, and service-oriented architecture is essential.

Skills

Backend Engineering

Infrastructure Engineering

Distributed Systems

Scalable APIs

Cloud-native Infrastructure

Real-time Serving

ML Infrastructure

GPU Orchestration

Service-oriented Architecture

Deployment Pipelines

System Observability

LLM Infrastructure

Model Inference

Tools

PyTorch

vLLM

Job description

DevHub is building scalable LLM infrastructure to power large-scale inference workloads. You will work with cross-functional teams to improve reliability, latency, and efficiency of distributed AI systems in a fast-growing environment.

We seek a senior backend/infrastructure engineer with 8+ years' experience in distributed systems, scalable APIs, and cloud-native infrastructure. Expertise in ML infrastructure, GPU orchestration, and SOA is essential; PyTorch and vLLM experience is a plus.