Member of Technical Staff, Production

Ashby, Inc.

Palo Alto (CA)

On-site

USD 180,000 - 280,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Health coverage
401(k) match
Meals and snacks
Discretionary productivity budget

Job summary

Inco AI is seeking founding members to own the production infrastructure behind its inference platform. You will ensure reliable, efficient operation of optimized inference engines on large GPU clusters serving customer traffic 24/7, spanning Kubernetes, orchestration, traffic management, deployments, observability, and capacity planning.

You will work with researchers and engineers to run at scale, with exposure to LLMs a plus.

Qualifications

  • Experience operating large-scale, always-on production infrastructure.
  • Strong debugging across applications, infrastructure, networking and distributed systems.
  • End-to-end systems thinking: ingress, routing, compute and customer delivery.
  • Automate recurring operational work rather than accepting toil.
  • Thrives in an early-stage startup with ambiguity and action bias.
  • Familiarity with AI/ML workflows and agent-based pipelines is a plus.

Responsibilities

  • Own production infrastructure for the inference platform.
  • Run highly optimized inference engines on large GPU clusters.
  • Manage Kubernetes, orchestration, traffic management and deployments.
  • Build observability with metrics, logs, dashboards and alerts.
  • Plan capacity and enable self-healing of systems.
  • Collaborate with researchers and engineers to scale workloads.

Skills

Linux
Networking
Containers
Python
Bash
Kubernetes
Distributed systems

Tools

Terraform
Prometheus
Grafana
GitOps

Job description

About the role

We are looking for founding members to own the production infrastructure behind Inco's inference platform. You will take highly optimized inference engines and make them run reliably and efficiently on large GPU clusters serving customer traffic 24/7. The work spans Kubernetes, orchestration, traffic management, deployments, observability, and capacity, with the goal of infrastructure that ships continuously without interrupting customers and recovers from failures on its own.

You bring deep expertise in distributed production infrastructure, and you will work closely with the researchers and engineers who build our inference engines to learn what it takes to run them at scale. Exposure to LLMs is a plus. Our organization is flat, and you will carry ideas from inception through deployment.

About you
  • You have operated large-scale, always-on production infrastructure and understand what it takes to keep complex distributed systems reliable

  • You are exceptional at debugging complex failures across applications, infrastructure, networking, and distributed systems

  • You think about systems end-to-end: how traffic enters the system, gets routed and scheduled, reaches compute, and ultimately serves the customer

  • You automate recurring operational work rather than accepting it as permanent toil

  • You are drawn to the risk and upside of an early-stage startup

  • You embrace ambiguity and have a strong bias for action

  • You operate with a tech-lead mentality regardless of title or years of experience

  • You treat AI as a force multiplier: fluent with coding agents and modern AI/ML workflows, you fold them into how you ship, and you are willing to push the boundary of what’s possible with agents

Requirements
  • Strong generalist engineering ability across Linux, networking, containers, Python, Bash, and distributed systems

  • Deep hands-on experience operating Kubernetes in production

  • Experience operating high-throughput, highly available production systems at significant scale

  • Strong understanding of cluster orchestration, scheduling, service discovery, load balancing, and traffic management

  • Experience building production observability using metrics, logs, dashboards, and actionable alerting

  • Strong understanding of networking fundamentals and how network behavior affects distributed applications

  • Experience designing safe deployment and rollout strategies for always-on production systems

Bonus
  • Experience operating large-scale AI systems, ML platforms, data platforms, or other compute-intensive distributed systems

  • Experience operating GPU clusters or AI/ML workloads

  • Familiarity with GPU environments including NVIDIA drivers, CUDA, NCCL, and InfiniBand/RDMA

  • Familiarity with LLM inference and serving stacks such as PyTorch, vLLM, SGLang, or TensorRT-LLM

  • Experience running bare-metal GPU fleets across multiple infrastructure providers

  • Experience with infrastructure-as-code tools such as Terraform

Compensation & Benefits
  • Highly competitive salary and equity

  • Comprehensive health coverage for you and your dependents

  • Strong 401(k) match

  • Meals and snacks on us + discretionary productivity budget

Strong preference for in-person in Palo Alto or Brooklyn.

About Inco

Inco AI (inco.ai) builds the agent-native inference platform for tomorrow’s token economics, making advanced AI fast enough for real workflows and affordable enough to use at scale. We’re the researchers and engineers behind AWQ, DFlash, and TensorRT-LLM, bringing frontier research into production.

Our platform ranked #1 in output speed on Artificial Analysis for every model we launched with, and our DFlash models have surpassed 10 million downloads on Hugging Face. Backed by B Capital, E14 Fund, and Tectonic Ventures, we serve customers across serverless, dedicated, and local AI deployments.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Member of Technical Staff, Developer Infrastructure & Agent Automation
Member of Technical Staff, Developer Infrastructure & Agent Automation

Inco AI • Palo Alto (CA)

On-site
USD 180,000 - 320,000
Salary & equity
Health coverage
401(k) match
+1
Member of Technical Staff, General
Member of Technical Staff, General

Ashby, Inc. • Palo Alto (CA)

On-site
USD 180,000 - 300,000
Competitive salary & equity
Health coverage for you and dependents
Strong 401(k) match
+1
Member of Technical Staff, Inference Systems
Member of Technical Staff, Inference Systems

Ashby, Inc. • Palo Alto (CA)

On-site
USD 180,000 - 280,000
Competitive salary
Equity
Health coverage
+2
Member of Technical Staff, Inference Research
Member of Technical Staff, Inference Research

Ashby, Inc. • Palo Alto (CA)

On-site
USD 200,000 - 350,000
Health coverage
401(k) match
Meals and snacks
+2
Member of Technical Staff, Developer Infrastructure & Agent Automation
Member of Technical Staff, Developer Infrastructure & Agent Automation

Ashby, Inc. • Palo Alto (CA)

On-site
USD 180,000 - 240,000
Highly competitive salary and equity
Comprehensive health coverage for you
Strong 401(k) match
+1
Member of Technical Staff, Kernel Performance & Automation
Member of Technical Staff, Kernel Performance & Automation

Ashby, Inc. • Palo Alto (CA)

On-site
USD 180,000 - 280,000
Highly competitive salary and equity
Comprehensive health coverage for you/
Senior Production Infra Engineer — AI Inference & GPUs
Senior Production Infra Engineer — AI Inference & GPUs

Ashby, Inc. • Palo Alto (CA)

On-site
USD 180,000 - 280,000
Health coverage
401(k) match
Meals and snacks
+1
Inference Engineer - Member of Technical Staff
Inference Engineer - Member of Technical Staff

Designworks Talent • Bellevue (WA)

Hybrid
USD 150,000 - 230,000
Medical, dental, and vision insurance
401(k) with company match
Paid holidays
(Staff / Senior Staff) AI Infra Engineer, Inference Infrastructure
(Staff / Senior Staff) AI Infra Engineer, Inference Infrastructure

Together AI • San Francisco (CA)

On-site
USD 250,000 - 320,000
Startup equity
Health insurance
Competitive benefits
(Senior/Junior AI Inference Reliability Engineer, Compute Infra
(Senior/Junior AI Inference Reliability Engineer, Compute Infra

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Equity
Health insurance