Senior Site Reliability Engineer

Hard Rock Digital

United States

Hybrid

USD 140,000 - 190,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Hybrid / remote work environment
Global brand with startup culture

Job summary

Hard Rock Digital is seeking an experienced Site Reliability Engineer to ensure the availability and performance of high-traffic Java applications. You will manage distributed systems, implement observability with Grafana/Prometheus, and lead incident response across production and staging environments.

Responsibilities include deploying monitoring stacks, optimizing JVM-based services, and integrating AI/LLM workflows for automated alerting and runbooks.

Qualifications

  • 5+ years in SRE/DevOps or similar infrastructure roles managing large-scale, high-availability production systems.

Responsibilities

  • Ensure availability, reliability, and performance of high-traffic Java-based applications in distributed environments.
  • Troubleshoot and resolve complex issues across production and non-production environments.
  • Conduct pre- and post-deployment performance testing and monitoring.
  • Deploy and manage Grafana, Prometheus, Loki, Mimir, and Alloy for monitoring, logging, and alerting.
  • Design, build, and operate agentic AI workflows for alert triage and incident summaries.

Skills

SRE/DevOps experience
Kubernetes
Grafana/Prometheus
Observability
Python/Bash/Go
CI/CD pipelines
AI/LLM tooling
Incident response

Education

BS in Computer Science or related field

Tools

Kubernetes
Kubectl/Helm
Grafana
Prometheus/Loki
ArgoCD
Terraform/Terragrunt
Ansible

Job description

  • Ensure availability, reliability, and performance of high-traffic Java-based applications in distributed environments
  • Troubleshoot and resolve complex issues across production and non-production environments
  • Conduct pre- and post-deployment performance testing and monitoring
  • Optimize Java application performance through JVM tuning, efficient resource utilization, and horizontal scaling
  • Deploy and manage Grafana, Prometheus, Loki, Mimir, and Alloy for monitoring, logging, and alerting
  • Implement observability strategies, dashboards, alerts, and log queries
  • Integrate AI/ML models for anomaly detection, predictive alerting, intelligent alert correlation, and noise reduction
  • Design, build, and operate agentic AI workflows for alert triage, root cause analysis, runbook execution, and incident summarization
  • Develop tool-calling LLM agents integrating with Kubernetes, Grafana, Jira, Slack, and PagerDuty
  • Build and maintain MCP servers and integrations exposing internal systems to AI agents
  • Evaluate and operationalize LLM frameworks and orchestration platforms for production agentic systems
  • Implement guardrails, evaluation harnesses, and feedback loops for safe and improving AI outputs
  • Champion AI-assisted development and operations practices across engineering
  • Support incident response, conduct post-mortems, identify root causes, and document lessons learned
  • Build AI-augmented automation, self-service tools, and chatbot interfaces for operational workflows
  • Measure and report toil-reduction metrics
  • Collaborate with developers, architects, data/ML engineers, DevOps, and NOC teams
  • Communicate SRE practices, AI capabilities, and operational insights to technical and non-technical stakeholders
  • Degree in Computer Science or a related field, or equivalent professional experience
  • 5+ years in SRE, DevOps, or similar infrastructure roles with experience managing large-scale, high-availability production systems
  • 3+ years hands-on experience managing production Kubernetes clusters, including deep understanding of architecture, networking, storage, and security
  • Experience with cluster autoscaling (Karpenter), upgrades, and multi-cluster management
  • Proficiency with kubectl, Helm, Kubernetes operators, and container orchestration troubleshooting
  • Advanced expertise with the Grafana observability stack, including dashboards, alerting, visualization, and Grafana Alloy
  • Proficiency in PromQL and experience with Loki for log aggregation and analysis
  • Hands-on experience managing Java-based applications in distributed environments, including JVM tuning and optimization
  • Cloud platform expertise; AWS preferred, with GCP or Azure also valued
  • Familiarity with Infrastructure as Code tools such as Terraform/Terragrunt or Ansible
  • ArgoCD proficiency for GitOps workflows and continuous deployment
  • Strong scripting abilities in Python, Bash, or Go, with experience building CI/CD pipelines and deployment automation
  • Proven track record with on-call rotations, incident response, and root cause analysis
  • 1+ years of practical experience building or operating AI/LLM-powered tools, agents, or workflows in a production or production-adjacent context
  • Ability to design agentic systems using tool calling, retrieval-augmented generation (RAG), or multi-step reasoning
  • Experience integrating LLM APIs such as Anthropic Claude, OpenAI, or open-source models into backend services or automation pipelines
  • Familiarity with at least one agentic orchestration framework or workflow engine, such as LangChain, LangGraph, CrewAI, n8n, Temporal, or equivalent
  • Understanding of prompt engineering best practices, including structured outputs, system prompts, and few-shot examples
  • Familiarity with AI-assisted coding tools such as Claude Code, Codex, or Cursor
  • Experience building or consuming MCP (Model Context Protocol) servers
  • Awareness of AI safety, hallucination mitigation, and human-in-the-loop design patterns
  • Strong written and verbal communication skills
  • Ability to mentor junior team members on traditional SRE practices and AI-driven approaches
  • Competitive pay and benefits
  • Flexible vacation allowance
  • A hybrid / remote working environment
  • Startup culture backed by a secure, global brand
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Engineer – AI Monitoring & Control Capabilities
Senior Engineer – AI Monitoring & Control Capabilities

Bank of America Corporation • Jacksonville (FL)

On-site
USD 150,000 - 210,000
Senior Engineer – AI Monitoring & Control Capabilities
Senior Engineer – AI Monitoring & Control Capabilities

Bank of America Corporation • Richmond (VA)

On-site
USD 180,000 - 240,000
AI Engineer
AI Engineer

Teserac • Santa Clara (CA)

On-site
USD 140,000 - 210,000
Health Care Plan
Paid Time Off
Free Food
+2
Senior Engineer – AI Monitoring & Control Capabilities
Senior Engineer – AI Monitoring & Control Capabilities

Bank of America Corporation • Pennington (NJ)

On-site
USD 180,000 - 240,000
AI-Driven SRE Automation Engineer
AI-Driven SRE Automation Engineer

Compunnel Inc. • Austin (TX)

Hybrid
USD 130,000 - 180,000
Senior Engineer – AI Monitoring & Control Capabilities
Senior Engineer – AI Monitoring & Control Capabilities

Bank of America Corporation • Jersey City (NJ)

On-site
USD 180,000 - 240,000
Staff Site Reliability Engineer
Staff Site Reliability Engineer

Ethos Life • New York (NY)

On-site
USD 180,000 - 240,000
401(k) retirement plan
Company equity
Paid time off and holidays
+1
Senior AI Engineer, Release Engineering
Senior AI Engineer, Release Engineering

Intercontinental Exchange • Atlanta (GA)

On-site
USD 140,000 - 200,000
Lead Software Engineer - Python - GenAI
Lead Software Engineer - Python - GenAI

JPMorganChase • Plano (TX)

Hybrid
USD 150,000 - 190,000
REMOTE Senior AI/ML Engineer - SaaS
REMOTE Senior AI/ML Engineer - SaaS

CyberCoders, Inc. • United States

Remote
USD 180,000 - 240,000
Remote work
Unlimited vacation
Comprehensive benefits
+2