Site Reliability Engineer Onsite Lahore PKR Salary

HR POD Careers; Lahore; On-site; PKR 3,000,000 - 4,200,000; Full time

Job summary

HR POD Careers in Lahore seeks an experienced SRE to own and optimize production Linux GPU clusters and AI infrastructure. You will troubleshoot complex distributed systems, GPUs, network and storage issues, and steward high reliability across multi-node environments.

The role emphasizes automation, IaC with Ansible/Terraform, and modern tooling like Grafana/Prometheus. Expect shift-based hours, collaboration with Platform teams, and ongoing performance improvements.

Qualifications

• 5+ years building and operating production Linux GPU clusters or AI infra.
• Bachelor's degree in Computer Science or equivalent.
• Strong Linux troubleshooting across OS, networking, storage, and performance.

Responsibilities

• Own reliability, availability, and performance of production Linux GPU clusters.
• Lead end‑to‑end troubleshooting of distributed systems, GPU nodes, and storage.
• Configure workload managers (Kubernetes, Slurm) and support services.
• Automate provisioning, image deployment, and remediation with IaC.
• Build observability and alerting using Grafana, Prometheus, Loki; enable self‑healing.

Skills

Linux systems administration

SRE

Kubernetes

GPU infrastructure

Python

IaC (Ansible/Terraform)

Observability (Grafana/Prometheus)

HPC / Slurm

NVIDIA GPU stack / NCCL

Networking / HPC storage

Education

Bachelor's degree in Computer Science

Tools

Ansible

Terraform

Grafana

Prometheus

Slurm

Job description

Job Description:

Requirements

• 5+ years of experience in systems, infrastructure, or SRE engineering, operating production systems at scale.
• Bachelor's degree in Computer Science.
• Deep Linux troubleshooting skills across the OS, networking, storage, and performance, with hands‑on experience working as root on production systems.
• Experience with Ubuntu is highly relevant, as it is used almost exclusively.
• Hands‑on experience operating GPU servers in production, including troubleshooting driver, device, and hardware‑level issues, rather than only the workloads running on top of them.
• Practical network troubleshooting experience, including diagnosing physical‑layer faults.
• Strong automation mindset with programming skills in Python or a comparable language.
• Experience with configuration management, node provisioning, and infrastructure‑as‑code (IaC) using Ansible, Terraform, or similar tools.
• Experience building observability and alerting solutions using Grafana and Prometheus.
• Experience operating GPU clusters or AI infrastructure at production scale.
• Production experience with Kubernetes or Slurm; experience with both is a bonus.
• Background in HPC or research computing.
• Familiarity with the NVIDIA GPU stack, InfiniBand/RDMA, and NCCL.
• Experience with CLI‑based AI coding agents such as Claude Code, rather than browser‑based assistants alone.
• Contributions to open‑source projects within the cloud‑native, HPC, or AI infrastructure ecosystem.

Responsibilities

• Own the reliability, availability, and performance of production Linux GPU clusters, covering the operating system, drivers, GPUs, high‑speed networking, and storage.
• Lead deep, end‑to‑end troubleshooting of complex distributed systems, GPU nodes, networking, and storage issues.
• Troubleshoot and resolve GPU rail and NCCL performance issues across multi‑GPU and multi‑node collective communication paths.
• Diagnose network faults end to end, including configuration, routing, and physical‑layer issues such as cabling, transceivers, and link errors.
• Configure and maintain workload managers that schedule customer jobs, including Kubernetes, Slurm, or both, along with the identity, storage, and networking services they depend on.
• Build automation and tooling to eliminate operational toil, using a modern language such as Python or Go to design solutions, review implementations, and redirect approaches when needed.
• Use AI‑assisted engineering tools such as Claude to accelerate automation, runbook development, and incident analysis.
• Automate provisioning, image deployment, configuration, and remediation using Ansible and infrastructure‑as‑code.
• Design and operate observability using Grafana, Prometheus, and Loki, while tuning alerts for meaningful signals and building self‑healing capabilities that reduce the need for human intervention.
• Lead incident response, on‑call activities, blameless postmortems, and reliability improvements that maintain customer SLAs.
• Partner with Platform and Systems Engineering teams on capacity planning, rollouts, and continuous improvement.

Working Hours

8 PM - 4 AM