Infra DevOps and Backend Engineer

GMI Cloud, Inc; Mountain View (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

GMI Cloud, Inc. seeks an Infrastructure Backend Engineering Development Engineer to design, build, and maintain scalable AI infrastructure. You will focus on cloud computing, distributed systems, and DevOps to enable efficient AI infrastructure operations.

The role involves designing infra for large-scale training/inference, automating GPU resource provisioning, and building observability solutions to ensure reliability and performance across GPU clusters.

Qualifications

• Bachelor’s degree in Computer Science or related field.
• Proficiency in at least one programming language (Golang, Python, Bash).
• Experience with OpenStack and Kubernetes orchestration.
• Automation and CI/CD pipelines with Ansible, Jenkins, GitLab CI.
• Telemetry/observability with Prometheus, Grafana, Mimir.
• Networking, security, and GPU cluster tuning.
• Deploying GPU clusters and managing GPU workloads.
• Secret management with HashiCorp Vault.
• Experience with distributed storage (Vastdata, Weka, DDN, Ceph).
• Strong system thinking and DevOps/cloud-native expertise.

Responsibilities

• Design, implement, and maintain AI/ML infrastructure for large-scale training and inference.
• Develop automation pipelines for resource provisioning and scheduling.
• Develop telemetry and observability solutions to monitor hardware performance and cluster health.
• Optimize infrastructure for high-throughput data transfer and low-latency communication.
• Manage infrastructure security, access controls, and compliance for on-prem GPU clusters.
• Collaborate with engineering teams to configure GPU clusters and resources.
• Document architecture, deployment procedures, and best practices.
• Stay current with GPU tech and integrate new hardware/software as appropriate.

Skills

Golang

Python

Bash

OpenStack

Kubernetes

CI/CD

Ansible

Jenkins

GitLab CI

Prometheus

Grafana

Mimir

Networking

Security

Performance tuning

GPU workloads

HashiCorp Vault

Vastdata

Weka

DDN

Ceph

System design

DevOps

Cloud-native architectures

Education

Bachelor’s degree in Computer Science

Job description

GMI Cloud is a fast-growing, AI-native infrastructure company delivering high-performance GPU compute, inference services, and infrastructure for AI agents. Following 8x ARR growth, GMI Cloud continues to scale rapidly across the U.S. and APAC. As a Reference Platform NVIDIA Cloud Partner (NCP) and a validated leading NCP across both markets, we power production AI for leading AI-native companies including Fireworks AI, Cartesia, Reflection, and OpenRouter. From large-scale compute to optimized inference and agentic workloads, GMI Cloud gives AI teams the infrastructure they need to build, deploy, and scale on one unified cloud. One cloud for compute, inference, and agents.

Role Overview

We are seeking a talented and highly skilled Infrastructure Backend Engineering Development Engineer to design, build, and maintain the scalable infrastructure that supports GMI AI/ML initiatives. The ideal candidate will have a strong background in cloud computing, distributed systems, and DevOps practices to enable efficient AI infrastructure operations.

Responsibilities

• Design, implement, and maintain AI/MLinfrastructure optimized for large-scale training and inference.
• Develop automation pipelines for GPU/CPU resource provisioning and workload scheduling using DevOps best practices and methodologies.
• Develop observability and telemetry solutions to pro-actively monitor hardware performance, utilization, and health to ensure cluster reliability and efficiency.
• Optimize infrastructure for high-throughput data transfer and low-latency communication.
• Manage infrastructure security, access controls, and compliance standards for on-prem GPU cluster environments.
• Collaborate with relevant engineering teams to configure and troubleshoot GPU clusters and hardware resources.
• Document infrastructure architecture, deployment procedures, automation workflow, and operational best practices.
• Stay current with the latest GPU technology developments, infrastructure engineering and integrate new hardware/software solutions as appropriate.

Qualifications

• Bachelor’s degree in Computer Science or related field.
• Proficiency in at least one programming language (Golang, Python, Bash) with strong coding practices and system design skills.
• Extensive experience with infrastructure orchestration platforms, especially OpenStack and Kubernetes.
• Strong proficiency in automation, configuration management, and CI/CD pipelines using Ansible, Jenkins, GitLab CI, or similar.
• Proven experience implementing telemetry and observability solutions (e.g. Prometheus, Grafana, Mimir and related technologies).
• Knowledge of networking, security, and performance tuning in GPU clusters.
• Hands-on experience in deploying GPU clusters and managing GPU workloads.
• Experience with secret management using HashiCorp Vault.
• Experience with distributed and high performance storage systems (e.g. Vastdata, Weka, DDN, Ceph)
• Strong system thinking and abstraction skills, capable of designing complex distributed systems from an end-to-end perspective.
• Strong understanding of DevOps principles, automation, and cloud-native architectures.

Meeting every qualification is not required—if you’re excited about this role, we’d love to hear from you. We believe diverse perspectives and experiences strengthen our team.