Specialist, Site Reliability Engineering

Merck Gruppe - MSD Sharp & Dohme

Hyderabad

Hybrid

INR 1,500,000 - 3,000,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Merck & Co., Inc. in Hyderabad seeks a Specialist in Site Reliability Engineering to keep the inference platform running and to build observability, capacity dashboards, and automated operations for production models.

You will implement SLOs, run on-call, and collaborate with hardware and security teams in a globally distributed IT landscape. The role requires 3–6 years in full‑stack development or related SRE/DevOps experience, strong scripting in Python/Bash, and hands-on Kubernetes,

Qualifications

  • Bachelor's degree in information technology, Computer Science or related field.
  • 3 to 6 years of hands-on experience working in full-stack development.

Responsibilities

  • Build observability for the inference layer, from GPU to gateway: GPU health and utilisation (DCGM, including XID and ECC errors), and engine metrics such as time to first token, inter-token latency, queue depth, KV cache use, preemptions, error rates and token throughput.
  • Run synthetic end-to-end probes that check the content of model responses, not only the HTTP status, since some serving failures return HTTP 200 with empty output.
  • Create the capacity and status dashboards that show what runs where and how much capacity is free.
  • Implement the SLOs, error budgets and alerts that the lead defines; run on-call and incident response; facilitate post-incident reviews.
  • Handle incidents, problems and changes through IT service management (ServiceNow or similar).
  • Automate operations: health checks, self-healing, safe upgrades, and the signals that tell the gateway when to spill traffic over to cloud models. Coordinate node draining and maintenance with the hardware and cluster team.
  • Run capacity and resilience tests and keep runbooks and the support model up to date.
  • Work with the hardware and cluster team on hardware faults and cluster issues, and with security on access and logging.

Skills

SRE / DevOps
Monitoring & incident mgmt
Python scripting
Bash scripting
LLM serving engines

Education

Bachelor's degree in information technology or Computer Science

Tools

Prometheus
Grafana
Kubernetes
Linux
ServiceNow

Job description

Job Description
Specialist, Site Reliability Engineering
The Opportunity
  • Based in Hyderabad, join a global healthcare biopharma company and be part of a 130- year legacy of success backed by ethical integrity, forward momentum, and an inspiring mission to achieve new milestones in global healthcare.
  • Be part of an organisation driven by digital technology and data-backed approaches that support a diversified portfolio of prescription medicines, vaccines, and animal health products.
  • Drive innovation and execution excellence. Be a part of a team with passion for using data, analytics, and insights to drive decision-making, and which creates custom software, allowing us to tackle some of the world's greatest health threats.

Our Technology Centers focus on creating a space where teams can come together to deliver business solutions that save and improve lives. An integral part of our company’s IT operating model, Tech Centers are globally distributed locations where each IT division has employees to enable our digital transformation journey and drive business outcomes. These locations, in addition to the other sites, are essential to supporting our business and strategy.

A focused group of leaders in each Tech Center helps to ensure we can manage and improve each location, from investing in growth, success, and well-being of our people, to making sure colleagues from each IT division feel a sense of belonging to managing critical emergencies. And together, we must leverage the strength of our team to collaborate globally to optimize connections and share best practices across the Tech Centers.

Role Overview

You keep the inference platform running. You build the monitoring, alerting, capacity dashboards and incident handling, and the automation that lets a small team support production models reliably.

What will you do in this role:
  • Build observability for the inference layer, from GPU to gateway: GPU health and utilisation (DCGM, including XID and ECC errors), and engine metrics such as time to first token, inter-token latency, queue depth, KV cache use, preemptions, error rates and token throughput.
  • Run synthetic end-to-end probes that check the content of model responses, not only the HTTP status, since some serving failures return HTTP 200 with empty output.
  • Create the capacity and status dashboards that show what runs where and how much capacity is free.
  • Implement the SLOs, error budgets and alerts that the lead defines; run on-call and incident response; facilitate post-incident reviews.
  • Handle incidents, problems and changes through IT service management (ServiceNow or similar).
  • Automate operations: health checks, self-healing, safe upgrades, and the signals that tell the gateway when to spill traffic over to cloud models. Coordinate node draining and maintenance with the hardware and cluster team.
  • Run capacity and resilience tests and keep runbooks and the support model up to date.
  • Work with the hardware and cluster team on hardware faults and cluster issues, and with security on access and logging.
What should you have
  • Bachelor's degree in information technology, Computer Science or related field
  • 3 to 6 years of hands-on experience working in full-stack development
Required Skills
  • DevOps or platform engineer running production services.
  • Hands-on Prometheus and Grafana (or equivalent), alerting and on-call practices.
  • Kubernetes and Linux operations; scripting in Python or Bash.
  • Familiarity with LLM serving engines (vLLM, SGLang or Triton) and their metrics.
  • Infrastructure as code and CI/CD.
Primary skills (must have)
  • Site Reliability Engineering (SRE) / DevOps / Platform Engineering
  • Prometheus & Grafana (or equivalent monitoring and observability tools)
  • Monitoring, Alerting and Incident Management practices.
  • Kubernetes Administration & Operations.
  • Linux System Administration / Operations
  • Python or Bash Scripting for automation and operational tasks.
  • LLM Serving Engines / Inference Infrastructure such as vLLM, SGLang, or Triton
  • Infrastructure as Code (IaC).
  • CI/CD Pipelines and Deployment Automation
Secondary skills (Nice to have)
  • Experience operating GPU clusters (NVIDIA DCGM, GPU Operator) or HPC environments.
  • Experience with ServiceNow incident, problem and change processes.
  • Cybersecurity: security monitoring and logging for production platforms, Kubernetes security (RBAC, network policies, admission control), vulnerability management, and handling incidents together with a security operations team.
Who we are:

We are known as Merck & Co., Inc., Rahway, New Jersey, USA in the United States and Canada and MSD everywhere else. For more than a century, we have been bringing forward medicines and vaccines for many of the world's most challenging diseases. Today, our company continues to be at the forefront of research to deliver innovative health solutions and advance the prevention and treatment of diseases that threaten people and animals around the world.

What we look for:

Imagine getting up in the morning for a job as important as helping to save and improve lives around the world. Here, you have that opportunity. You can put your empathy, creativity, digital mastery, or scientific genius to work in collaboration with a diverse group of colleagues who pursue and bring hope to countless people who are battling some of the most challenging diseases of our time. Our team is constantly evolving, so if you are among the intellectually curious, join us—and start making your impact today.

Additional location(s) Compensation Information:

Bash (Scripting Language), CI/CD, Deployment Automation, DevOps, Grafana, Infrastructure As Code (IaC), Kubernetes, Large Language Model (LLM) Fine-Tuning, Linux System Administration, Platform Engineering, Prometheus (Software), Python (Programming Language), Site Reliability Engineering

Secondary Language(s) Job Description

#MSDHYDIT

Requisition ID: R427210

Employee Status: Regular

Relocation: Domestic

VISA Sponsorship: No

Travel Requirements: No Travel Required

Flexible Work Arrangements: Hybrid

Shift: Not Indicated

Valid Driving License: No

Hazardous Material(s): n/a

Job Posting End Date: 11/24/2026

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Specialist, Site Reliability Engineering
Specialist, Site Reliability Engineering

MSD • Hyderabad

On-site
INR 1,800,000 - 2,800,000
Senior Specialist, Inference Engineering
Senior Specialist, Inference Engineering

MSD KK • Hyderabad

Hybrid
INR 3,500,000 - 6,000,000
Specialist, Inference Engineering
Specialist, Inference Engineering

MSD • Hyderabad

Hybrid
INR 1,800,000 - 3,000,000
Hybrid work arrangements
Senior Specialist, Inference Engineering
Senior Specialist, Inference Engineering

MSD • Hyderabad

On-site
INR 2,400,000 - 4,200,000
Senior Specialist, Software Engineering
Senior Specialist, Software Engineering

MSD KK • Hyderabad

Hybrid
INR 1,500,000 - 2,500,000
Specialist, Inference Engineering
Specialist, Inference Engineering

MSD KK • Hyderabad

Hybrid
INR 1,800,000 - 3,200,000
Senior Specialist, AI Evals Engineer
Senior Specialist, AI Evals Engineer

MSD KK • Hyderabad

Hybrid
INR 1,800,000 - 4,000,000
Hybrid working model
Global projects
International environment
+2
Senior Specialist, AI Evals Engineer
Senior Specialist, AI Evals Engineer

JRNLClub • Hyderabad

Hybrid
INR 4,000,000 - 7,500,000
Specialist, AI Evals Engineer
Specialist, AI Evals Engineer

3640 MSD Hyderabad Global Capabilities Center India Private Limited • Hyderabad

Hybrid
INR 1,500,000 - 2,100,000
Specialist, AI Evals Engineer
Specialist, AI Evals Engineer

Merck Sharp & Dohme Corp. • Hyderabad

Hybrid
INR 1,800,000 - 3,200,000