Senior AI Inference & Runtime Platform Engineer - Remote

Jobgether; United States; Remote; USD 140,000 - 230,000; Full time

Benefits offered by this job

Salary

Equity

Bonus

Healthcare

Flexible time off

Remote work

Company summits

Open-source impact

Job summary

Jobgether is partnering with a US‑based team seeking a Senior Software Engineer (AI Inference & Runtime Platform). You will own scalable AI infrastructure, from high‑performance model inference to secure sandbox execution, across Rust, Python and cloud platforms.

Expect to work on Kubernetes clusters, GPU resources and open‑source contributions. You should have 5+ years in production systems, strong Rust or willingness to use Rust, and hands‑on experience with GPU orchestration, cryptographic

Qualifications

• At least 5 years delivering production systems with a systems programming language, preferably Rust.
• Experience operating Kubernetes workloads, controllers or operators, autoscaling, and node lifecycle management.
• Knowledge of secure AI‑agent execution, least‑privilege access, and audit trails.
• Experience deploying open‑weight LLM inference infrastructure and production endpoints.

Responsibilities

• Manage model-serving infra for open‑weight LLMs, including upgrades and canaries.
• Operate Kubernetes infra for inference and sandbox workloads with GPU scheduling.
• Own the data plane end‑to‑end with reliable recovery procedures.
• Develop secure agent sandbox runtime and host control plane.
• Build FastAPI control-plane services and IaC dashboards.

Skills

Rust

Go/C/C++/Zig

Kubernetes

Distributed systems

Python

Async runtimes

Open-source

Education

Bachelor's degree

Tools

Kubernetes

Terraform/OpenTofu

Bicep

OpenTelemetry

AWS/Azure/GCP

Job description

Jobgether is partnering with a US‑based team seeking a Senior Software Engineer (AI Inference & Runtime Platform). You will own scalable AI infrastructure, from high‑performance model inference to secure sandbox execution, across Rust, Python and cloud platforms.

Expect to work on Kubernetes clusters, GPU resources and open‑source contributions. You should have 5+ years in production systems, strong Rust or willingness to use Rust, and hands‑on experience with GPU orchestration, cryptographic