Senior Software Engineer (AI Inference & Runtime Platform)

Intuit, Inc.

Bellevue (WA)

Remote

USD 140,000 - 230,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health insurance
Equity
Flexible PTO
Bonus eligibility
Fully remote culture

Job summary

AZX seeks a senior systems engineer to own the AI execution layer, including the inference control plane, agent-sandboxing, and related Kubernetes orchestration. You will implement scalable, secure, low-latency runtimes for open-weight models across GPU clouds and customer-managed clusters.

You'll write Rust and Kubernetes controllers, collaborate on FastAPI endpoints, and design the fork engine and multi-substrate model lifecycle. Remote-friendly, Seattle cluster noted.

Qualifications

  • 5+ years shipping production systems in a systems language (Rust preferred).
  • Experience operating Kubernetes workloads with controllers, autoscaling, and node lifecycle.
  • Threat-model isolation boundaries (namespaces, cgroups, seccomp) with secure sandbox practices.
  • Hands-on deploying open-weight LLM serving infrastructure (vLLM/SGLang or similar).
  • Performance-focused distributed systems experience; measure before optimizing.
  • Proficiency in Rust (tokio), Python/FastAPI, Kubernetes operators, KEDA, Karpenter, GPU plugins, with willingness to learn others.
  • Familiarity with isolation tech (Firecracker, Kata, gVisor), secrets management and egress control, and stateful hosting (vector stores, graph stores).
  • Cloud/GPU substrates (AWS/Azure/GCP) with Terraform/OpenTofu and OpenTelemetry.

Responsibilities

  • Manage the serving tier for open-weight models: deployment, config, cold-start, per-model SLOs, upgrades.
  • Administer Kubernetes layer for inference and sandbox workloads: operators/CRDs, autoscaling, GPU scheduling, node lifecycle.
  • Own the stateful data plane end-to-end with tested restore procedures.
  • Oversee sandbox runtime and host-side control plane: lifecycle, exec, snapshot/fork, teardown, metering, audit.
  • Direct the FastAPI control-plane services, Terraform/OpenTofu, Bicep, dashboards.
  • Run the layer the backend services team builds on and debug their services while reviewing dashboards.
  • Manage open-source posture: OSS-compliant builds, docs, reproducible builds.

Skills

Rust
Go
C/C++
Zig

Education

Bachelor's Degree
Master's degree

Tools

Kubernetes
Terraform/OpenTofu
Bicep
OpenTelemetry
KEDA
Karpenter
Vault

Job description

Provn is helping AZX recruit for the following role

About AZX:

Our mission is to accelerate positive impact in critical industries through AI transformation. We specialize in physics-informed ML and enterprise AI solutions that directly address climate and sustainability challenges.

We're growing quickly and already work with category-leaders in real estate (CBRE), energy (LevelTen Energy), logistics (Flexe) and utilities.

We're a public benefit corporation, founded in 2024, and have been profitable from inception.

We work on challenges in clean energy, decarbonization, climate risk, energy systems, and global economics. We're building our company for long-term success and aim to create the ultimate place to work for those passionate about AI and making a positive impact.

About the role:

You will be responsible for owning the layer where AI work physically happens: the machines, the isolation boundary, and the models running on them. This role anchors on two systems. The first is our inference control plane: open-weight models and custom task-model zoos, hosted and operated across managed GPU clouds and customer-managed Kubernetes clusters, with scale-to-zero economics, cold-start discipline, and per-token cost accounting that stays correct even when a client disconnects mid-stream, along with the Kubernetes layer those workloads live on (operators, autoscaling, node lifecycle). The second is our agent-sandboxing platform: hardware-isolated microVMs for running untrusted, agent-generated code securely and compliantly by construction, where agents operate with least privilege, never see a credential, and a human gates anything that writes to a system of record. You'll write Rust in the morning, a Kubernetes controller after lunch, and a FastAPI control-plane endpoint before you go home, building the fork engine, the guest agent, and the multi-substrate model lifecycle. We're looking for individuals who've built this class of stack (an inference-serving or serverless-GPU platform), operated it hard at scale, or ideally both.

Responsibilities
  • Manage the serving tier for open-weight models: engine deployment and configuration, cold-start strategy, per-model SLOs, and upgrade/canary discipline.

  • Administer the Kubernetes layer for inference and sandbox workloads: operators and CRDs, autoscaling (KEDA/Karpenter-class), GPU scheduling and sharing, and node lifecycle.

  • Own the stateful data plane end to end with restore procedures that are regularly tested.

  • Oversee the sandbox runtime and its host-side control plane: lifecycle, exec, snapshot/fork, teardown, metering, and the threat model of the isolation boundary.

  • Direct the FastAPI control-plane services, Terraform/OpenTofu, Bicep, and the dashboards.

  • Run the layer the backend services team builds on, expect to debug into their services, and expect them to read your dashboards.

  • Manage the open-source posture: build to OSS standards and release as it matures, with reviewed PRs, real docs, and reproducible builds.

Required Qualifications
  • 5+ years of shipping production systems in a systems language. Rust is the house language, but polyglots are welcome: deep Go, C/C++, or Zig with genuine appetite for Rust counts. Async runtimes, memory-safety discipline, and debugging at the syscall boundary should be familiar territory.

  • Operated Kubernetes workloads that other people depended on: controllers or operators, scheduling, autoscaling, node lifecycle. You've been paged, and the experience changed how you build.

  • Strong ability to threat-model isolation boundaries (namespaces, cgroups, seccomp, hypervisors), including identifying what an untrusted guest could observe, forge, or exhaust, and applying security best practices for agentic execution: least privilege, no credentials in the sandbox, audit trails, and human approval on write actions.

  • Hands-on experience deploying or operating open-weight LLM serving infrastructure (vLLM/SGLang or similar), including packaging models into reliable, metered production endpoints.

  • Performance discipline in distributed systems: you measure before you optimize, and you can tell the story of a latency you killed with the numbers attached.

  • Practical depth in some of our core stack: Rust (tokio), Python/FastAPI, Kubernetes operators (controller-runtime/Kubebuilder/CRDs), KEDA, Karpenter, GPU device plugins/DRA, with a genuine willingness to research your way into the rest.

  • Familiarity with isolation technology (Firecracker, Kata, gVisor, or comparable), secrets management and egress control (Vault/KMS-class), and hosting stateful systems (vector stores like pgvector/Qdrant, graph stores like Neo4j) with backup and failover discipline.

  • Comfort operating across cloud and GPU substrates (AWS/Azure/GCP plus managed GPU clouds) using infrastructure-as-code (Terraform/OpenTofu, Bicep) and observability tooling (OpenTelemetry).

  • Bachelor's Degree; Master's is a plus.

Additional Information
  • Must be able to travel 2x/year for company summits

  • Applicants must be currently authorized to work in the United States on a full-time basis.

  • We are unable to sponsor or take over sponsorship of employment visas at this time.

Benefits

Competitive early-stage startup compensation (based on capabilities, experience, and location). Bonus eligibility and equity. Health insurance with meaningful coverage for dependents. Flexible PTO. Fully remote culture with a cluster of teammates in Seattle. Join a fast-growing, profitable, mission-driven company with industry-leading clients leading AI transformation in critical industries.

Estimated annual OTE

$140-230k.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Software Engineer (AI Inference & Runtime Platform)
Senior Software Engineer (AI Inference & Runtime Platform)

AZX • Seattle (WA)

Hybrid
USD 150,000 - 190,000
Health insurance
Equity
Fully remote culture with Seattle team
+2
Senior Software Engineer (AI Inference & Runtime Platform)
Senior Software Engineer (AI Inference & Runtime Platform)

AZX • United States

Remote
USD 180,000 - 275,000
Equity
Health insurance
Flexible PTO
+1
Forward Deployed AI Engineer (Senior)
Forward Deployed AI Engineer (Senior)

Intuit, Inc. • Bellevue (WA)

Remote
USD 140,000 - 230,000
Health insurance
Equity
Remote‑friendly culture
+1
Senior Backend Engineer (AI Platform)
Senior Backend Engineer (AI Platform)

Intuit, Inc. • Bellevue (WA)

Remote
USD 140,000 - 230,000
Equity
Health insurance
Flexible PTO
+2
Senior AI Inference & Runtime Platform Engineer
Senior AI Inference & Runtime Platform Engineer

AZX • Seattle (WA)

Hybrid
USD 150,000 - 190,000
Health insurance
Equity
Fully remote culture with Seattle team
+2
Senior ML Engineer (Client Solutions)
Senior ML Engineer (Client Solutions)

Intuit, Inc. • Bellevue (WA)

Remote
USD 140,000 - 230,000
Health insurance
Equity & Bonus eligibility
Flexible PTO
+1
Senior Product Engineer (AI, Full-Stack)
Senior Product Engineer (AI, Full-Stack)

Intuit, Inc. • Bellevue (WA)

Remote
USD 140,000 - 230,000
Equity
Bonus eligibility
Health insurance
+2
Forward Deployed AI Engineer (Senior)
Forward Deployed AI Engineer (Senior)

AZX • United States

Remote
USD 140,000 - 230,000
Health insurance
Equity
Fully remote culture
+2
Software Engineer (Backend-Focused)
Software Engineer (Backend-Focused)

AZX • United States

Remote
USD 140,000 - 210,000
Health insurance
Equity
Flexible PTO
+2
Senior ML Engineer (Energy & Utilities)
Senior ML Engineer (Energy & Utilities)

Intuit, Inc. • Bellevue (WA)

Remote
USD 140,000 - 230,000
Bonus eligibility
Equity
Health insurance for dependents
+2