Founding ML Infra Engineer — Equity & Open-Source AI

Peano AI

Palo Alto (CA)

On-site

USD 180,000 - 260,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Peano AI is building the Agent Cloud infrastructure and seeks a Founding ML Infrastructure Engineer to design and optimize high‑performance model serving for large‑scale open‑source models in Palo Alto.

You will own end‑to‑end ML systems, work with accelerators like TPUs/GPUs, improve latency and throughput, and collaborate with research and product teams to ship production‑grade infrastructure.

Qualifications

  • Strong experience in ML systems, distributed systems, or high-performance computing.
  • Experience optimizing inference or training workloads for large models.
  • Familiarity with TPUs, GPUs, or other accelerators.
  • Experience with CUDA, Triton, NCCL, JAX/XLA, PyTorch internals, vLLM, SGLang, or TensorRT‑LLM.
  • Strong systems debugging skills.
  • Comfort working across model code, runtime, infrastructure, and product requirements.
  • High ownership in an early-stage startup environment.

Responsibilities

  • Optimize large-scale LLM inference and serving systems.
  • Improve tokens per second, latency, throughput, and cost efficiency.
  • Work on serving infrastructure for open-source models across accelerators.
  • Improve batching, scheduling, KV cache management, memory usage, and accelerator utilization.
  • Support long-context inference up to 1M context.
  • Debug performance bottlenecks across model execution, runtime, networking, and infrastructure.
  • Work with frameworks such as JAX/XLA, PyTorch, vLLM, SGLang, TensorRT-LLM, or related systems.
  • Collaborate with the application team to optimize infrastructure for agentic workloads.
  • Help turn research prototypes into reliable production systems.

Skills

ML systems
Distributed systems
High-performance computing
Inference optimization
Training optimization
TPUs
GPUs

Tools

CUDA
Triton
NCCL
JAX/XLA
PyTorch internals
vLLM
SGLang
TensorRT-LLM

Job description

Peano AI is building the Agent Cloud infrastructure and seeks a Founding ML Infrastructure Engineer to design and optimize high‑performance model serving for large‑scale open‑source models in Palo Alto.

You will own end‑to‑end ML systems, work with accelerators like TPUs/GPUs, improve latency and throughput, and collaborate with research and product teams to ship production‑grade infrastructure.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Founding Machine Learning Infrastructure Engineer
Founding Machine Learning Infrastructure Engineer

Peano AI • Palo Alto (CA)

On-site
USD 180,000 - 260,000
Senior ML Infra Engineer for Large-Scale Mid-Training & RL
Senior ML Infra Engineer for Large-Scale Mid-Training & RL

Peano AI • Palo Alto (CA)

On-site
USD 200,000 - 260,000
Founding Infra Engineer for GPU Training & Inference
Founding Infra Engineer for GPU Training & Inference

Peano AI • Palo Alto (CA)

On-site
USD 140,000 - 210,000
Founding Mid-Training/RL Infrastructure Engineer
Founding Mid-Training/RL Infrastructure Engineer

Peano AI • Palo Alto (CA)

On-site
USD 200,000 - 260,000
Founding ML Systems Engineer - End-to-End Infrastructure AI
Founding ML Systems Engineer - End-to-End Infrastructure AI

Meter • San Francisco (CA)

Hybrid
USD 200,000 - 320,000
Founding Infrastructure Engineer - GPU & AI Ops
Founding Infrastructure Engineer - GPU & AI Ops

Peanoai • Palo Alto (CA)

On-site
USD 180,000 - 260,000
Member of Technical Staff
Member of Technical Staff

Harrison Clarke • San Francisco (CA)

On-site
USD 180,000 - 280,000
Senior Staff ML Infra Engineer for Scalable AI
Senior Staff ML Infra Engineer for Scalable AI

Pinterest, Inc. • San Francisco (CA)

Hybrid
USD 245,000 - 429,000
Equity
Staff ML Engineer: Production-Scale AI & Equity
Staff ML Engineer: Production-Scale AI & Equity

Hive • San Francisco (CA)

On-site
USD 200,000 - 300,000
Staff ML Infrastructure Architect
Staff ML Infrastructure Architect

Quince • Palo Alto (CA)

On-site
USD 218,000 - 285,000