Founding Infrastructure Engineer Engineering · Remote | West Coast (GMT -7 to GMT -8)

Gradiant; United States; Remote; USD 180,000 - 260,000; Full time

Benefits offered by this job

Equity stake

Remote-friendly

Flexible hours

Health benefits

Job summary

Gradiant is seeking a founding infrastructure engineer to own our GPU fleet and the serving stack, ensuring reliability as we scale from private beta to real traffic. You will design, build and operate production GPU infrastructure for high-throughput, low-latency inference while working closely with researchers to push experiments into production.

This is a seed-stage environment requiring hands-on ownership, strong fundamentals in Kubernetes, Linux, networking, and distributed systems, and a

Qualifications

• 10+ years of hands-on infrastructure work in production systems.

Responsibilities

• Design, build and run production GPU infrastructure for high-throughput, low-latency inference of open models.
• Run and tune inference serving stacks in production (vLLM, SGLang, TensorRT-LLM or in-house).
• Own reliability end-to-end including monitoring, capacity planning and incident response.
• Collaborate with researchers to move experiments into production.
• Secure GPUs through partnerships to ensure scalable access.

Skills

Kubernetes

Linux

Networking

Distributed systems

GPU infrastructure

SRE/On-call

CUDA

NCCL

RDMA/InfiniBand

Tools

CUDA

NCCL

RDMA/InfiniBand

Bare-metal

Job description

Own our complete inference infrastructure end to end, from GPU fleet to serving stack, as we go from private beta to real traffic and scale beyond.

Why this role exists

We only succeed if the infrastructure underneath is fast, efficient and reliable. Right now nobody fully owns it. We need a founding engineer who has run infrastructure at scale and has also built from scratch at a seed-stage startup. This person will own the GPU fleet, the serving stack and its reliability as we go from private beta to real traffic, and scale beyond.

What you will do

• Design, build and run production GPU infrastructure for high-throughput, low-latency inference of open models
• Run and tune inference serving stacks in production, whether that's vLLM, SGLang, TensorRT-LLM or our own
• Own reliability end to end, including monitoring, capacity planning and incident response
• Work closely with researchers so that hardware and software experiments make it into production
• Work with partners to secure the GPUs we need to run models

Problems you may work on

• Optimizing routing across GPUs based on which deployments have LoRA adapters in memory or based on KV cache locality
• Scheduling training runs to utilize spare capacity in periods of lower inference traffic
• Ensuring that networking across multiple cloud providers is secure and fast
• Building observability that shows exactly where cost and latency go across the hot path
• Experimenting with changes to our inference engine to optimize cache hits for request routing
• Building deployment pipelines that reliably handle rolling deployments without reducing serving capacity, and roll-back automatically when things are going wrong
• Setting up on-call and incident processes that suit a small team today and still hold up at 100x the traffic

What we are looking for

• 10+ years of hands-on infrastructure work. That can be SRE, platform or deep backend infra. The title matters less than having owned production systems that had to stay up.
• Seed-stage experience. You've been a founder or a founding or very early engineer, and you've built from nothing with a tiny team.
• Experience at real scale. You've also worked somewhere with serious traffic, data or user volume, and you've seen what breaks. You have worked on a service in the hot-path receiving a lot of requests that needed to have at least four nines of availability.
• Recent GPU or AI infrastructure work. Inference serving is ideal. GPU clusters, HPC, or ML platform work at an AI-focused company also counts.
• Strong fundamentals: Kubernetes, Linux, networking and distributed systems. NCCL, RDMA/InfiniBand, CUDA or bare-metal experience is a bonus.
• Based in in UCT-7 or -8 time zone, working remotely on West Coast hours. We will split on-call between US and EU timezones so that we don't have to be up at night when things break.
• Low ego and hands-on. You write clearly, you ship, and you'd rather fix the thing than present about fixing it.

This role is not for you if

• You'd rather direct infrastructure than build it yourself. This is a keyboard-first role, and you'll carry the pager.
• Your career has been almost entirely at big tech companies, and you haven't worked in the scrappy, nothing-exists-yet setting of a seed-stage startup.
• You've only worked at small startups and haven't yet operated systems at real scale.
• You've never operated a service that needed to be highly available.
• Your GPU or ML experience is mostly from before 2020, or it's mainly research and benchmarking without production operations.
• You're looking for a short stint. We want someone to build this with us for years.
• Conference talks and thought leadership excite you more than a clean incident postmortem.