Senior Production Infra Engineer — AI Inference & GPUs

Ashby, Inc.

Palo Alto (CA)

On-site

USD 180,000 - 280,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health coverage
401(k) match
Meals and snacks
Discretionary productivity budget

Job summary

Inco AI is seeking founding members to own the production infrastructure behind its inference platform. You will ensure reliable, efficient operation of optimized inference engines on large GPU clusters serving customer traffic 24/7, spanning Kubernetes, orchestration, traffic management, deployments, observability, and capacity planning.

You will work with researchers and engineers to run at scale, with exposure to LLMs a plus.

Qualifications

  • Experience operating large-scale, always-on production infrastructure.
  • Strong debugging across applications, infrastructure, networking and distributed systems.
  • End-to-end systems thinking: ingress, routing, compute and customer delivery.
  • Automate recurring operational work rather than accepting toil.
  • Thrives in an early-stage startup with ambiguity and action bias.
  • Familiarity with AI/ML workflows and agent-based pipelines is a plus.

Responsibilities

  • Own production infrastructure for the inference platform.
  • Run highly optimized inference engines on large GPU clusters.
  • Manage Kubernetes, orchestration, traffic management and deployments.
  • Build observability with metrics, logs, dashboards and alerts.
  • Plan capacity and enable self-healing of systems.
  • Collaborate with researchers and engineers to scale workloads.

Skills

Linux
Networking
Containers
Python
Bash
Kubernetes
Distributed systems

Tools

Terraform
Prometheus
Grafana
GitOps

Job description

Inco AI is seeking founding members to own the production infrastructure behind its inference platform. You will ensure reliable, efficient operation of optimized inference engines on large GPU clusters serving customer traffic 24/7, spanning Kubernetes, orchestration, traffic management, deployments, observability, and capacity planning.

You will work with researchers and engineers to run at scale, with exposure to LLMs a plus.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Inference Engineer - Production Focus (Remote)
Senior Inference Engineer - Production Focus (Remote)

Vcluster • San Francisco (CA)

Hybrid
USD 190,000 - 220,000
Equity
Health, dental, vision, and life保险
Flexible working schedule
+1
Senior AI Inference Reliability Engineer - GPU & Kubernetes
Senior AI Inference Reliability Engineer - GPU & Kubernetes

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Equity
Health insurance
Senior AI Inference Reliability Engineer - GPU Compute Infra
Senior AI Inference Reliability Engineer - GPU Compute Infra

Together Computer Inc • San Francisco (CA)

On-site
USD 250,000 - 320,000
Equity
Health insurance
Competitive benefits
Member of Technical Staff, Production
Member of Technical Staff, Production

Ashby, Inc. • Palo Alto (CA)

On-site
USD 180,000 - 280,000
Health coverage
401(k) match
Meals and snacks
+1
Founding Infrastructure Engineer, AI Inference Platform
Founding Infrastructure Engineer, AI Inference Platform

Collective Intuition, Inc. • New York (NY)

Hybrid
USD 120,000 - 200,000
Head of AI Inference Infrastructure
Head of AI Inference Infrastructure

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000
Kubernetes Platform Engineer for AI Inference
Kubernetes Platform Engineer for AI Inference

present of AI • San Francisco (CA)

On-site
USD 180,000 - 240,000
Inference Platform Engineer — Production-Scale AI
Inference Platform Engineer — Production-Scale AI

Designworks Talent • Bellevue (WA)

Hybrid
USD 150,000 - 230,000
Medical, dental, and vision insurance
401(k) with company match
Paid holidays
GPU AI Infra Reliability Engineer
GPU AI Infra Reliability Engineer

Together AI • San Francisco (CA)

On-site
USD 250,000 - 320,000
Startup equity
Health insurance
Competitive benefits
ML Infrastructure Engineer — Scalable AI Inference
ML Infrastructure Engineer — Scalable AI Inference

Abridge • San Francisco (CA)

On-site
USD 170,000 - 210,000
Generous Time Off
Comprehensive Health Plans
Generous 401(k) Matching
+4