AI Infra Reliability Engineer – GPU/HPC & Observability

MARGO; United States; Remote; USD 70,000 - 88,000; Full time

Job summary

MARGO is seeking a Site Reliability/Platform Engineer to help build and operate a large AI infrastructure with monitoring, diagnosis, and remediation for production incidents. You will collaborate with multiple engineering teams to keep services healthy and scalable.

The role includes on-call rotations, implementing observability, and lifecycle management across environments and countries, with emphasis on stability, security, and performance.

Qualifications

• Experience with large-scale AI infrastructure and GPU/HPC environments.
• Strong Linux system administration and scripting.
• Experience with monitoring and logging tools (Prometheus, Grafana, Elastic).
• Knowledge of networking concepts (DNS, TCP/IP, VLANs, IPv6).
• Familiarity with IaC (Ansible, Salt, AWX).
• Experience with CI/CD pipelines (GitLab).

Responsibilities

• Build a large AI infrastructure with monitoring, diagnosis, and remediation of production incidents.
• Troubleshoot high-impact production issues in collaboration with other engineering teams
• Participate in an on-call rotation to handle incidents and ensure service continuity
• Implement and maintain observability solutions to monitor AI infrastructure and application health
• Contribute to AI infrastructure lifecycle management across different environments and countries
• Promote and apply best practices in terms of stability, resiliency, scalability, and security
• Maintain clear technical documentation for tools and procedures
• Collaborate closely with development teams to ensure infrastructure readiness- Participate in team rituals and knowledge-sharing initiatives

Skills

Go

Python

Bash

Linux

Networking

GitLab CI/CD

Prometheus

Grafana

Elastic

Ansible

Tools

Prometheus

Grafana

Elastic

Salt

AWX

MariaDB

GitLab

Job description

MARGO is seeking a Site Reliability/Platform Engineer to help build and operate a large AI infrastructure with monitoring, diagnosis, and remediation for production incidents. You will collaborate with multiple engineering teams to keep services healthy and scalable.

The role includes on-call rotations, implementing observability, and lifecycle management across environments and countries, with emphasis on stability, security, and performance.