Staff AI Platform Engineer: Scale LLM Pipelines

Fastino, Inc.; San Francisco (CA); On-site; USD 180,000 - 280,000; Full time

Job summary

Fastino, Inc. is hiring a Staff Software Engineer in San Francisco to define the platform's technical direction and scale our AI systems.

You will design and build the agent harness, runtime loop, and eval harnesses, while scaling training and inference pipelines for latency and cost efficiency. You will deploy into customer-managed AWS VPCs using Kubernetes and Terraform, and create secure sandboxed environments for agent workloads on Modal Sandboxes.

Qualifications

• 8+ years of hands-on software engineering, with leadership across a team or product.
• Strong architectural skills with a track record of scaling production systems.
• Production experience with agentic AI training/inference pipelines.
• Hands-on experience deploying software on AWS with Kubernetes and Terraform.
• Experience building sandboxed execution environments for agent workloads, ideally with Modal Sandboxes.
• Experience building with AI coding agents to produce production-quality code.
• Thrives in ambiguity and balances fast delivery with long-term robustness.
• Optional: agent harnesses or inference tooling experience.

Responsibilities

• Define the technical direction for Fastino’s platform and scale the architecture.
• Design and build the agent harness, runtime loop, tool calling, context management, and eval harnesses.
• Scale AI training and inference pipelines focusing on latency, throughput, and cost.
• Deploy the Fastino platform into customer-managed AWS VPCs using Kubernetes and Terraform.
• Create secure sandboxed environments for agent workloads on Modal Sandboxes.
• Deliver end-to-end features while solving complex, ambiguous problems.
• Collaborate with AI researchers to take models from research to production, including monitoring.
• Mentor teams, enforce coding standards, and shape best practices for AI coding agents.
• Engage with customers and translate feedback into product decisions.

Skills

Hands-on software eng

Software architecture

Production AI systems

Ambiguity tolerance

Tools

AWS

Kubernetes

Terraform

Modal Sandboxes

LangSmith

vLLM

Triton

Job description

Fastino, Inc. is hiring a Staff Software Engineer in San Francisco to define the platform's technical direction and scale our AI systems.

You will design and build the agent harness, runtime loop, and eval harnesses, while scaling training and inference pipelines for latency and cost efficiency. You will deploy into customer-managed AWS VPCs using Kubernetes and Terraform, and create secure sandboxed environments for agent workloads on Modal Sandboxes.