Senior Software Engineer, AI Platform

Provn; Seattle (WA); Remote; USD 180,000 - 260,000; Full time

Benefits offered by this job

Equity participation

Bonus eligibility based on performance

Comprehensive health insurance

Flexible paid time off

Job summary

AZX is seeking a Senior Software Engineer, AI Platform, to architect and manage core AI inference and runtime infrastructure. This full-time, remote role focuses on scalable deployment, secure code execution, and robust data planes, supporting clients across real estate, energy, logistics, and utilities.

The role emphasizes Rust, distributed systems, Kubernetes operators, and open-weight LLM serving with Terraform/OpenTofu/Bicep integration.

Qualifications

• 5+ years delivering production systems in a systems programming language, with significant Rust expertise.
• Strong experience managing production Kubernetes workloads, including autoscaling and GPU scheduling.
• Advanced security practices around isolation boundaries, namespaces, cgroups, seccomp, and hypervisors.

Responsibilities

• Manage and optimize the serving tier for open-weight AI models, including deployment, config, cold-start strategies and SLOs.
• Administer Kubernetes environments for inference and sandbox workloads, including operators and CRDs, autoscaling, and GPU scheduling.

Skills

Rust

Go

C/C++

Zig

Kubernetes

Python

FastAPI

Education

Bachelor’s degree

Master’s degree

Tools

Terraform

OpenTofu

Bicep

Kubebuilder

KEDA

Karpenter

OpenTelemetry

Job description

Senior Software Engineer, AI Platform - Summary

Join AZX as a Senior Software Engineer, AI Platform, and play a pivotal role in shaping the future of AI transformation for critical industries. In this full-time, remote role, you’ll architect and manage the core systems that power AI inference and runtime platforms, including advanced infrastructure for scalable deployment and secure code execution. AZX is dedicated to accelerating positive impact through AI, focusing on applications across clean energy, climate risk, and enterprise solutions. As a Senior Software Engineer, AI Platform, you’ll have the chance to work with leading industry clients and leverage your expertise in systems engineering, Kubernetes, and distributed computing to design robust, secure, and efficient platforms.

AZX is committed to diversity, equity, and inclusion, providing a collaborative environment for talented professionals who are passionate about AI and its real-world applications. If you are driven to build scalable, production-grade systems and thrive in a mission-driven organization, we encourage you to apply for this opportunity as a Senior Software Engineer, AI Platform.

Duties & Responsibilities

• Manage and optimize the serving tier for open-weight AI models, handling engine deployment, configuration, cold-start strategies, and per-model service level objectives (SLOs).
• Administer Kubernetes environments for inference and sandbox workloads, including development of operators, custom resource definitions (CRDs), autoscaling, GPU scheduling, and node lifecycle management.
• Own and maintain the stateful data plane, ensuring robust backup and restore procedures are implemented and regularly tested.
• Oversee the agent sandbox runtime and its control plane, focusing on secure lifecycle management, execution, snapshotting, teardown, metering, and enforcing isolation boundaries.
• Develop and maintain FastAPI control-plane endpoints and integrate with infrastructure tools such as Terraform, OpenTofu, and Bicep.
• Provide operational support for backend service teams, including debugging, dashboard creation, and performance monitoring.
• Contribute to open-source initiatives, ensuring code meets OSS standards, documentation is comprehensive, and builds are reproducible.

Salary & Benefits

• Competitive salary commensurate with experience and capabilities.
• Bonus eligibility based on performance metrics.
• Equity participation reflecting your contribution to the company’s growth.
• Comprehensive health insurance, including coverage for dependents.
• Flexible paid time off to support work-life balance.
• Fully remote work culture supporting nationwide collaboration.
• Opportunity to work with leading clients in real estate, energy, logistics, and utilities.
• Engage in company summits and professional development initiatives.

Qualifications & Requirements

• Minimum of 5 years’ experience delivering production systems in a systems programming language, with significant expertise in Rust preferred; strong backgrounds in Go, C/C++, or Zig also considered.
• Demonstrated experience managing production Kubernetes workloads, including controllers, operators, scheduling, autoscaling, and node lifecycle management.
• Advanced knowledge of security practices around isolation boundaries, namespaces, cgroups, seccomp, and hypervisors.
• Hands-on experience deploying or operating open-weight LLM serving infrastructure (e.g., vLLM, SGLang) with an emphasis on reliability and metered endpoints.
• Proven ability to measure and optimize distributed system performance, with a data-driven approach to latency reduction and system improvements.
• Practical experience with Rust (tokio), Python/FastAPI, Kubernetes operators (controller-runtime, Kubebuilder, CRDs), KEDA, Karpenter, and GPU device plugins/DRA.
• Familiarity with isolation technologies such as Firecracker, Kata, or gVisor, as well as secrets management and hosting stateful systems (pgvector, Qdrant, Neo4j) with reliable backup and failover strategies.
• Experience operating across multiple cloud and GPU environments (AWS, Azure, GCP, managed GPU clouds) and using infrastructure-as-code tools (Terraform, OpenTofu, Bicep) and observability platforms (OpenTelemetry).
• Bachelor’s degree required; Master’s degree is a plus.
• Eligibility to work full-time in the U.S. is required.

Ideal Candidate Snapshot

• Deep systems engineering expertise, especially with Rust and distributed computing environments.
• Proven track record in designing, deploying, and operating large-scale, production-grade AI or serverless-GPU platforms.
• Strong security orientation with practical experience in sandboxing and agentic execution.
• Quick learner, adaptable to new technologies and capable of diving into complex technical challenges.
• Collaborative mindset, comfortable working across teams and supporting open-source contributions.
• Compelled by AZX’s mission to use AI for positive, real-world impact in critical industries.

Other Relevant Information

• The Senior Software Engineer, AI Platform role is remote, with periodic company-wide summits to foster team cohesion and knowledge sharing.
• AZX is an equal opportunity employer, committed to building a diverse and inclusive workforce. All qualified applicants will receive consideration without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, disability, veteran status, or other protected characteristics.