Senior Reliability Engineer

Hims & Hers; United States; Remote; USD 140,000 - 190,000; Full time

Benefits offered by this job

Generous PTO

Full healthcare

401(k) with matching

Remote-first

Stock options

Employee discounts

Technology stipend

HSA/FSA options

Job summary

Hims & Hers is seeking a Senior Reliability Engineer to own reliability across critical customer journeys. You will build observability, tooling, and automation to catch problems before users are affected, and you’ll drive incident response, RCA drafting, and post-mortems with a data-driven mindset.

We value AI-first approaches, strong collaboration with Security and product teams, and hands-on work with FireHydrant, Datadog, Jira, and Confluence. Remote-first, US-based teams welcome applicants.

Qualifications

• Experience owning reliability outcomes for production systems
• Proven ability to define instrument and monitor SLOs and golden signals
• Strong incident management and post-mortem practices
• Production experience with AWS, Kubernetes, Terraform, and relational databases

Responsibilities

• Define and instrument SLOs, golden signals, and monitors for critical journeys
• Lead capacity planning and resilience work for peak load and failures
• Mature incident response, runbooks, and post-mortem action tracking
• Automate OE workflows with AI agents and tooling
• Collaborate with security and product teams on anomaly detection and response

Skills

SRE

Observability

Incident management

AWS

Kubernetes

Tools

Datadog

FireHydrant

Jira

Confluence

Terraform

PostgreSQL

Job description

• We’re looking for a Senior Reliability Engineer to make Hims & Hers systems measurably more reliable: building the observability, tooling, and automation that catch problems before people do
• Recent work at this level includes preparing our stack for 32x baseline traffic during high-stakes seasonal surges with zero customer-facing issues and improved P95 latency, turning databases into a paved platform with consistent observability and guardrails, and building AI agents that pull FireHydrant, Datadog, Jira, and Confluence into a single incident picture
• We use AI first: every engineer gets a Claude Enterprise license, and we expect you to use it as a core part of how you investigate, build, and write, not as an afterthought
• Own reliability for Tier 1 customer journeys: Define and instrument SLOs, golden signals, and business-level monitors for the journeys that matter most (checkout, telehealth visits, prescription fulfillment), so that degradation is detected by systems rather than by customers or support tickets. Move our “detected by monitors vs. detected by humans” ratio in the right direction and be able to show it
• Engineer for peak load and failure: Lead capacity and resilience work for high-stakes events and steady-state growth: load-test design, second-by-second analysis of prior events, database and backend bottleneck investigation, and hardening across caching, GraphQL, VPC capacity, and vendor rate limits. Validate before game day, not during it
• Build incident response as software: Mature FireHydrant, Datadog, and Jira into one connected pipeline: automated incident and RCA ticket creation, SLO burn-rate and composite alerting with deduplication, Tier 1 alert routing, and enforced post-mortem action tracking. Author and maintain the runbooks and severity standards that make any responder effective on any service
• Automate operational excellence with AI: Build and operate agents and tooling that reduce manual OE work: OER report generation, RCA drafting, stale action-item detection, monitor and runbook gap detection, and OpenClaw agents wired to Datadog and FireHydrant for first-pass incident triage. Ship these as reusable capabilities, not personal scripts
• Be the deep-debugging expert, and make teams better at it: Teams own debugging their own services, but you are the person they pull in when a problem crosses boundaries: silent service-to-service failures, anomalous traffic, or regressions that span frontend, API, mesh, and database layers
• Bring that depth to the hardest cases yourself, then turn what you learn into runbooks, tooling, and pairing so teams can catch and resolve the next one on their own. Partner with Security and product teams on anomaly detection and response, weighing engineering cost and user impact alongside the benefit of any control
• Maintain the tooling that informs Operational Excellence reviews: Produce the tooling that powers the metrics and narrative that go into bi-weekly VP-level OE reviews and the monthly cross-engineering OER, and drive the resulting action items to closure
• Raise the bar for others: Document what you build, onboard teammates to roll it out, and coach engineers across squads on SLOs, blameless post-mortems, and on-call practice

Benefits

• Generous PTO: Take the time you need, when you need it - including generous parental leave
• Full healthcare: High-coverage medical, dental & vision coverage for individuals and families
• Retirement planning: Take advantage of our 401(k) plan including contribution matching
• Work from anywhere: We are a remote-first company, so you can work from anywhere you like in the uS
• Robust compensation: We offer competitive salary bands and stock options
• Employee discount: Employees can take advantage of product discounts
• Utility stipend: A extra $75 each month to cover extra cell phone, internet, or data usage
• Spending accounts: Options for additional HSA and FSA plans to help toward healthcare costs

Strong software engineering fundamentals. You solve reliability problems by writing code and building tooling, and you’re comfortable reading application code across the stack to find the real causeDaily, practical use of AI coding and analysis tools (Claude, Cursor, or similar) to accelerate investigation, code, documentation, and reporting, and clear judgment about when to trust the output and when to verify itCommunication skills to explain risk, tradeoffs, and post-incident learnings to engineers and leadership alike, in writing and in review meetingsProduction experience with AWS, Kubernetes/EKS, Terraform, and PostgreSQL (RDS/Aurora)Hands-on depth in observability and SLO engineering: golden signals, burn-rate alerting, journey-level monitors, and turning noisy alert streams into actionable pages (Datadog preferred; Prometheus/Grafana and OpenTelemetry welcome)5+ years as a Software, SRE, Platform, or Infrastructure Engineer, with a track record of owning reliability outcomes for production systems that customers depend onExperience running or maturing incident management end to end: on-call design, escalation policies, incident command, blameless post-mortems, and action-item follow-through in a tool like FireHydrant or PagerDutyWe are committed to building a workforce that reflects diverse perspectives and prioritizes ethics, wellness, and a strong sense of belonging. If you’re excited about this role, we encourage you to apply—even if you’re not sure if your background or experience is a perfect matchExperience building AI agents or LLM-backed automation for operations: incident triage, RCA generation, alert correlation, or observability data analysis, ideally with MCP or tool-calling integrations against Datadog, FireHydrant, or JiraFamiliarity with service mesh (Istio) and its observability and traffic-management featuresBackground in a regulated or healthcare environment, where reliability and data handling carry patient-safety and compliance weightLoad-testing and performance-engineering experience at meaningful scale (k6 or similar), including validating systems to well above expected peakExperience designing vendor and partner escalation frameworks with defined severities and response SLAs