Observability & Scale Engineer

Kyndryl Inc.

Dallas (TX)

On-site

USD 115,000 - 219,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Annual bonus
Medical & dental
Paid time off
Retirement benefits

Job summary

Kyndryl Inc. is seeking an experienced engineer to own operational visibility and capacity planning across the full service stack of our AI infrastructure in Dallas.

You will design telemetry models, implement Prometheus and Grafana dashboards, monitor GPU utilization, and develop executive scorecards to optimize reliability and cost.

Qualifications

  • 4+ years in observability, performance engineering, capacity management, SRE, or infrastructure analytics.
  • Hands-on Prometheus, Grafana, time-series data, APIs, dashboarding, and alert engineering.
  • Ability to translate telemetry into operational, performance, and investment recommendations.

Responsibilities

  • Design a common telemetry model across power, cooling, compute, GPU, network, storage, Kubernetes and services.
  • Implement and maintain Prometheus, Grafana, and enterprise observability integration.
  • Monitor health, ECC conditions, throttling, thermals, link state, congestion, and SLOs.
  • Build topology-aware dashboards, alerts, baseline reports, and executive scorecards.
  • Baseline idle, peak, degraded, and recovery conditions and correlate GPU utilization with demand.
  • Forecast GPUs, fabric ports, storage bandwidth, rack power, and expansion triggers.

Skills

Prometheus
Grafana
Telemetry
Dashboarding
SRE
Kubernetes
OpenTelemetry
Cloud observability

Tools

DCGM
NVIDIA Run:ai
NCCL
Elastic
Splunk
Datadog
Dynatrace
VictoriaMetrics
Thanos

Job description

Who We AreAt Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward – always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities.The RoleAccountable for operational visibility and capacity planning across the full service stack. Gives leadership early warning of reliability and capacity risk and the evidence needed to optimize costly AI infrastructure and plan expansion.Key responsibilities· Design a common telemetry model spanning power, cooling, compute, GPU, NVLink, network, storage, Kubernetes, scheduler, workload, and service layers.· Implement and maintain DCGM or equivalent GPU telemetry, Prometheus, Grafana, vendor management telemetry, log pipelines, and enterprise observability integration.· Monitor health, ECC conditions, throttling, thermals, link state, retransmissions, congestion, queue depth, storage latency, workload demand, and service SLOs.· Build topology-aware dashboards, alerts, dependency views, baseline reports, and operational or executive scorecards.· Baseline idle, steady-state, peak, degraded, and recovery conditions and correlate GPU utilization with CPU, memory, network, storage, queue, and application demand.· Define evidence-based thresholds and reduce false positives through tuning, suppression, routing, and contextual enrichment.· Forecast GPUs, fabric ports, storage bandwidth, rack power, cooling, scheduler demand, and expansion triggers.· Identify stranded capacity, bottlenecks, noisy-neighbor conditions, low utilization, and workload-placement inefficiency and recommend corrective action..Kyndryl currently does not require employees to be fully vaccinated against COVID-19, however, if you are hired to work at a client, customer, or partner location, you may be required to show proof of vaccination to align with their respective COVID-19 vaccination policies. Those who believe they are eligible may apply for a medical or religious accommodation prior to the start of employment.Who You AreRequired Qualifications· 4+ years in observability, performance engineering, capacity management, SRE, or infrastructure analytics.· Hands-on Prometheus, Grafana, time-series data, APIs, dashboarding, and alert engineering.· Ability to translate telemetry into operational, performance, and investment recommendations.Preferred Skills· NVIDIA DCGM, NVIDIA Run:ai, NCCL performance, UFM, and Spectrum-X or InfiniBand telemetry.· NVIDIA GPU utilization and workload - efficiency analysis , topology-aware monitoring, and AI capacity modeling.· Cloud observability, HPC performance engineering, network telemetry, storage analytics, or enterprise capacity management.· OpenTelemetry, Elastic, Splunk, Datadog, Dynatrace, VictoriaMetrics, Thanos, or comparable platforms.· Slurm accounting, Kubernetes metrics, FinOps, or infrastructure demand forecasting.· SRE, observability platform, or cloud-native credentials.Travel RequirementsLimited nationwide travel may be required; percentage to be confirmed.The compensation range for the position in the U.S. is - $115,080 to $218,520 based on a full-time schedule. Your actual compensation may vary depending on your geography, job-related skills and experience. For part time roles, the compensation will be adjusted appropriately. The pay or salary range will not be below any applicable state, city or local minimum wage requirement. There is a different applicable compensation range for the following work locations:California (San Francisco Bay Area): $138,120 to $262,320California (All Other): $126,600 to $240,480Colorado: $115,080 to $218,520Massachusetts $115,080 to $240,480New York City: $138,120 to $262,320Washington: $126,600 to $240,480Washington DC: $126,600 to $240,480This position will be eligible for Kyndryl’s discretionary annual bonus program, based on performance and subject to the terms of Kyndryl’s applicable plans. You may also receive a comprehensive benefits package which includes medical and dental coverage, disability, retirement benefits, paid leave, and paid time off.Note: If this is a sales commission eligible role, you will be eligible to participate in a sales commission plan in lieu of the annual discretionary bonus program.Applications will be accepted on a rolling basis.Being YouDiversity is a whole lot more than what we look like or where we come from, it’s how we think and who we are. We welcome people of all cultures, backgrounds, and experiences. But we’re not doing it single-handily: Our Kyndryl Inclusion Networks are only one of many ways we create a workplace where all Kyndryls can find and provide support and advice. This dedication to welcoming everyone into our company means that Kyndryl gives you – and everyone next to you – the ability to bring your whole self to work, individually and collectively, and support the activation of our equitable culture. That’s the Kyndryl Way.What You Can ExpectWith state-of-the-art resources and Fortune 100 clients, every day is an opportunity to innovate, build new capabilities, new relationships, new processes, and new value. Kyndryl cares about your well-being and prides itself on offering benefits that give you choice, reflect the diversity of our employees and support you and your family through the moments that matter – wherever you are in your life journey. Our employee learning programs give you access to the best learning in the industry to receive certifications, including Microsoft, Google, Amazon, Skillsoft, and many more. Through our company-wide volunteering and giving platform, you can donate, start fundraisers, volunteer, and search over 2 million non-profit organizations. At Kyndryl, we invest heavily in you, we want you to succeed so that together, we will all succeed.Get Referred!If you know someone that works at Kyndryl, when asked ‘How Did You Hear About Us’ during the application process, select ‘Employee Referral’ and enter your contact's Kyndryl email address.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Validation & Automation Engineer
AI Validation & Automation Engineer

Kyndryl Inc. • Dallas (TX)

On-site
USD 144,000 - 273,000
Medical and dental coverage
Comprehensive benefits package
Observability & Scale Engineer
Observability & Scale Engineer

1000 Kyndryl, Inc. • Dallas (TX)

On-site
USD 115,000 - 219,000
Restaurant d'entreprise
Indemnités de travail à distance
NVIDIA AI Enterprise Forward Deployed Engineer
NVIDIA AI Enterprise Forward Deployed Engineer

Kyndryl Inc. • Dallas (TX)

On-site
USD 115,000 - 219,000
Medical coverage
Dental coverage
Retirement benefits
+3
AI Infrastructure Architect
AI Infrastructure Architect

Kyndryl Inc. • Dallas (TX)

On-site
USD 180,000 - 342,000
Annual bonus program
Comprehensive benefits package
AI Operational Readiness & Knowledge Lead
AI Operational Readiness & Knowledge Lead

Kyndryl • Dallas (TX)

On-site
USD 100,000 - 190,000
Medical and dental coverage
Disability benefits
Retirement benefits
+1
AI Platform Operations Engineer
AI Platform Operations Engineer

Kyndryl Inc. • Frisco (TX)

Hybrid
USD 144,000 - 273,000
Medical coverage
Dental coverage
Disability benefits
+3
AI Infrastructure Engineer
AI Infrastructure Engineer

1000 Kyndryl, Inc. • Dallas (TX)

On-site
USD 144,000 - 273,000
Annual bonus
Medical & dental
Retirement plan
+1
SOP, Knowledge & Readiness Lead
SOP, Knowledge & Readiness Lead

HITEC • San Francisco (CA), New York (NY)

On-site
USD 120,000 - 228,000
Medical insurance
Dental coverage
Disability coverage
+3
Observability & Scale Engineer
Observability & Scale Engineer

Kyndryl Inc. • Frisco (TX)

Hybrid
USD 115,000 - 219,000
Annual bonus
Comprehensive benefits
AI Platform Operations Engineer
AI Platform Operations Engineer

1000 Kyndryl, Inc. • Dallas (TX)

On-site
USD 144,000 - 273,000
Discretionary annual bonus
Comprehensive benefits package