Principal Inference Reliability Engineer

Parasail; California; On-site; USD 180,000 - 280,000; Full time

Job summary

Parasail is redefining AI infrastructure by enabling seamless deployment across a distributed network of GPUs. The Senior/Staff Inference Reliability Engineer owns end-to-end reliability of customer inference workloads and drives performance improvements across the serving path.

You will build telemetry, investigate complex latency issues, and lead incident response to ensure availability, latency, throughput, and cost targets are met while learning inference-specific technologies.

Qualifications

• 5+ years of experience in production engineering, site reliability engineering, infrastructure engineering, distributed systems, ML infrastructure, database reliability, or performance engineering.
• Demonstrated ownership of a critical production service or workload.
• Experience diagnosing complex latency, throughput, capacity, or reliability problems across multiple system layers.
• Strong software-engineering ability beyond infrastructure configuration and CI/CD automation.

Responsibilities

• Own the production health of customer inference workloads, including availability, request success, time to first token, inter-token latency, throughput, and operational efficiency.
• Establish clear service-level indicators, objectives, performance baselines, and escalation paths for production endpoints.
• Build telemetry, dashboards, alerts, and automated diagnostics to detect meaningful endpoint degradation early.
• Lead the investigation of latency, throughput, capacity, and reliability regressions across multiple components.
• Help lead customer-impacting incidents and translate findings into platform improvements and runbooks.

Skills

Kubernetes

Linux

Networking

Cloud infrastructure

Python/Go/Java/C++/Rust

Job description

Parasail is redefining AI infrastructure by enabling seamless deployment across a distributed network of GPUs. The Senior/Staff Inference Reliability Engineer owns end-to-end reliability of customer inference workloads and drives performance improvements across the serving path.

You will build telemetry, investigate complex latency issues, and lead incident response to ensure availability, latency, throughput, and cost targets are met while learning inference-specific technologies.