Sr Specialist System Engineering (4G/5G-ANT DevOps/Infra Engineer)

AT&T; Bengaluru; On-site; INR 4,000,000 - 6,500,000; Full time

Job summary

AT&T in Bengaluru is seeking an experienced DevOps/Infrastructure Lead to drive Kubernetes, CI/CD automation, and IaC across 4G/5G core simulations and production environments. You will design pipelines, manage Helm charts, and implement secure, scalable operations spanning on‑prem and cloud components.

Role requires deep Kubernetes expertise, strong scripting in Python/Ansible/Bash, and hands‑on with Vault, OpenStack HEAT, and Azure Bicep. Offshore collaboration with U.S.

Qualifications

• Hands‑on experience operating Kubernetes platforms and production‑grade CI/CD pipelines (Azure DevOps, GitHub/GitOps).
• Strong automation skills in Python, Ansible, and Bash for deployment orchestration and tooling.
• Deep experience with Helm 3 (OCI) packaging and release management; artifact publishing (JFrog Artifactory OCI).
• Expertise in secrets management (HashiCorp Vault + Ansible Vault), credential rotation, and secure repo practices.
• Experience with infrastructure provisioning and IaC (OpenStack HEAT and/or Azure Bicep) across hybrid environments.
• Strong troubleshooting and incident management skills across K8s clusters, networking, and stateful services.
• Working knowledge of advanced K8s networking/storage (Multus, SR‑IOV, MetalLB, Calico; Rook‑Ceph/local storage).
• Understanding of 5G/LTE telecom networking fundamentals and containerized NF operational needs (e.g., DPDK / SCTP / GTP considerations).

Responsibilities

• Serve as the primary offshore DevOps/Infra support lead for Automation Tools and 4G/5G Core Network Simulators across NPRD/Labs and PROD environments.
• Design, build, and maintain Azure DevOps CI/CD pipelines using a 3-tier architecture to automate packaging, publishing, and deployments.
• Develop and maintain Helm v3 (OCI) charts for application and infra components; package/publish artifacts to JFrog Artifactory OCI.
• Automate Kubernetes deployments via Python orchestration, including config substitution, sequencing, and post-deploy health checks.
• Own secrets management end-to-end: Ansible Vault encryption, Vault runtime retrieval, and token rotation with zero-plaintext controls.
• Provision and operate infrastructure: KVM/QEMU/libvirt VM lifecycle on Simzone hosts; implement SR‑IOV passthrough and host networking configurations.
• Author/maintain IaC using OpenStack HEAT and Azure Bicep.
• Configure advanced K8s networking and security: Multus NADs, MetalLB VIPs, Calico policies, SR‑IOV for performance I/O.
• Manage storage and stateful services: Rook‑Ceph provisioning, local-storage for single‑node clusters, PVC sizing, and storage class governance.
• Build/maintain large Ansible automation footprint for deployment, upgrades, backup/restore, vault operations, and compliance tasks.
• Operate AOSM lifecycle pipelines to publish NF definitions, build deployment configs, deploy to NAKS, and onboard to HCEV.
• Drive environment promotion and release consistency across lab/pre‑prod/prod with versioning and branching.
• Maintain per-instance configuration at scale using real-parameter/value files and enforce correctness across environments.
• Implement observability and verification: deploy kube-prometheus-stack, integrate DevOps metrics, and standardize health checks.
• Own certificate lifecycle management via cert-manager and Vault integration, including rotation workflows.
• Maintain supply-chain security artifacts (bom.yaml validation, images.json versions).
• Administer multi-repo governance across 30+ repos: branch protections, PR governance, and cross-repo dependency coordination.
• Build AI-assisted DevOps tooling to accelerate analysis, pipeline engineering, diagnostics, and self-healing operations.
• Provide incident response and operational troubleshooting: kubectl diagnosis, log analysis, NF status checks, backup/restore, and chaos testing support.

Skills

Kubernetes platforms

Azure DevOps

GitHub/GitOps

Python

Ansible

Bash

Helm 3 OCI

Tools

JFrog Artifactory OCI

HashiCorp Vault

Ansible Vault

OpenStack HEAT

Azure Bicep

Job description

Job Responsibilities

• Serve as the primary offshore DevOps/Infra support lead for Automation Tools and 4G/5G Core Network Simulators across NPRD/Labs and PROD environments.

• Design, build, and maintain Azure DevOps CI/CD pipelines using a 3‑tier (hyper→macro→micro) architecture to automate packaging, publishing, and deployments.

• Develop and maintain Helm v3 (OCI) charts for application and infra components; package/publish artifacts to JFrog Artifactory OCI.

• Automate Kubernetes deployments via Python orchestration (e.g., deploy.py), including config substitution, sequencing, and post‑deploy health checks.

• Own secrets management end‑to‑end: Ansible Vault encryption, HashiCorp Vault runtime retrieval (AppRole/KV), and token rotation with zero‑plaintext controls.

• Provision and operate infrastructure: KVM/QEMU/libvirt VM lifecycle on Simzone hosts; implement SR‑IOV passthrough and host networking configurations.

• Author/maintain Infrastructure‑as‑Code using OpenStack HEAT (on‑prem VM provisioning) and Azure Bicep (NAKS/AON resources).

• Configure advanced K8s networking and security: Multus NADs (multi‑NIC pods), MetalLB L2 VIPs, Calico policies, SR‑IOV for performance I/O.

• Manage storage and stateful services: Rook‑Ceph provisioning (RWX/RWO), local‑storage for single‑node clusters, PVC sizing, and storage class governance.

• Build/maintain large Ansible automation footprint (deployment, upgrades, backup/restore, mechIDmgmt, vault operations, compliance tasks).

• Operate AOSM lifecycle pipelines (nfPL0→nfPL4) to publish NF definitions, build deployment configs, deploy to NAKS, and onboard to HCEV.

• Drive environment promotion (NPRD→PROD): release branching, versioning updates, quarterly release coordination, and consistency across lab/pre‑prod/prod.

• Maintain per‑instance configuration at scale (20+ K8s instances) using real‑parameters-values.json / input‑values.json and enforce correctness across environments.

• Implement observability and verification: deploy kube‑prometheus‑stack, integrate DevOps metrics (DORA Build Events API), and standardize service health checks.

• Own certificate lifecycle management via cert‑manager and vault‑issuer patterns, including rotation workflows and Vault PKI integration.

• Maintain supply‑chain security artifacts (bom.yaml validation, images.json version tracking, reproducible builds with pinned versions).

• Administer multi‑repo governance (30+ repos across GitHub + ADO): branch protections, PR governance, and cross‑repo dependency coordination.

• Build AI‑assisted DevOps tooling (agentic workflow with specialized agents) to accelerate analysis, pipeline engineering, diagnostics, and self‑healing operations.

• Provide incident response and operational troubleshooting: kubectl diagnosis, log analysis, NF status checks, backup/restore execution, and chaos testing support.

Job Qualifications / Required Qualifications

• Hands‑on experience operating Kubernetes platforms and production‑grade CI/CD pipelines (Azure DevOps, GitHub/GitOps).

• Strong automation skills in Python , Ansible, and Bash for deployment orchestration and operational tooling.

• Deep experience with Helm 3 (OCI) packaging and release management; artifact publishing (JFrog Artifactory OCI).

• Expertise in secrets management (HashiCorp Vault + Ansible Vault), credential rotation, and secure repo practices.

• Experience with infrastructure provisioning and IaC (OpenStack HEAT and/or Azure Bicep) across hybrid environments.

• Strong troubleshooting and incident management skills across K8s clusters, networking, and stateful services.

• Working knowledge of advanced K8s networking/storage (Multus, SR‑IOV, MetalLB, Calico; Rook‑Ceph/local storage).

• Understanding of 5G/LTE telecom networking fundamentals and containerized NF operational needs (e.g., DPDK / SCTP / GTP considerations).

• IP/Networking/routing/firewalls/Ubuntu repo's packages knowledge and hands on experience.

Preferred Qualifications

• Experience operating AOSM pipelines (nfPL0→nfPL4) and NF onboarding workflows into NAKS/AON/HCEV environments.

• Experience managing KVM/QEMU/libvirt VM fleets on bare‑metal, including SR‑IOV passthrough and host networking.

• Experience with supply‑chain security artifacts (BOM generation/validation, digest pinning, reproducible builds).

• Proven ability to manage large‑scale repo/pipeline governance across many repos with strong PR/branch policy discipline.

• Experience building or integrating AI‑assisted DevOps workflows for diagnostics, automation, and operational self‑healing.

Additional Job Information

This is an offshore role that requires daily collaboration with U.S. stakeholders, including overlapping work hours to ensure effective partnership and meet business needs.

Weekly Hours: 40

Time Type: Regular

Location: IND:KA:Bangalore / Epip Area, Hoodi Village, Whitefield Rd - Eqp: Plot 111/112, Epip Area, Hoodi Village, Whitefield Road

AT&T and its subsidiaries are committed to equal employment opportunity. All hiring, promotion, and other employment decisions remain merit-based and free from discrimination on the basis of race, color, religion, religious creed, national origin, ancestry, age, sex, sexual orientation, gender, gender identity, gender expression, physical disability, mental disability, pregnancy, medical condition, genetic information, marital status, citizenship status, military status, veteran status, or any other characteristic protected by federal, state, or local laws. In addition, AT&T will provide reasonable accommodations to qualified individuals with disabilities. AT&T is a fair chance employer and does not initiate a background check until an offer is made. Click here to learn more or request an application accommodation here.