Staff Backend Observability Engineer (Remote-First)

Affirm, Inc.; Richmond (VA); Remote; USD 204,000 - 264,000; Full time

Benefits offered by this job

Health coverage for you and dependents

Monthly tech stipend

Flexible time off and holidays

Employee stock purchase plan (ESPP)

Job summary

AFFIRM is seeking a Staff Software Engineer to lead the Observability effort. You’ll own the telemetry pipeline architecture, balance reliability and cost, and mentor a team of engineers while partnering with infra and product teams.

You will drive log sampling, AI-assisted observability integration, and set a year-long technical strategy to deliver high-impact features and improve incident response. Remote-first with flexible in-person options across the US.

Qualifications

• 8+ years designing, developing and launching backend systems at scale.
• Experience with Python or Kotlin in production environments.
• Track record building highly available distributed systems using AWS, MySQL, Flink and Kubernetes.
• Built or operated telemetry systems (metrics/logs/traces) with OpenTelemetry, Kafka and Terraform.
• Strong viewpoint on observability practices, including wide events, sampling and SLOs.
• Led platform standard adoption across multiple teams with tooling and engagement.
• Delivered major features or deprecations via clear technical plans.
• Excellent communication and ability to operate across ambiguity.

Responsibilities

• Own the architecture of AFFIRM’s telemetry pipeline for 200+ TB/day.
• Design log sampling strategies to reduce volume and cost without losing signal.
• Shape observability within agentic development for AI coding agents.
• Set technical strategy and guide team on year-long initiatives.
• Collaborate with product management, design and analytics for sustainability and risk awareness.
• Advocate for scalable tech solutions and operational processes.
• Own team operations, monitoring, on-call readiness, and playbooks.
• Foster quality and ownership through design standards and code reviews.

Skills

Backend systems design

Distributed systems

Observability practices

Code quality & reviews

Mentoring/leadership

Cross-functional collaboration

Problem solving in ambiguity

Education

Bachelor’s degree or equivalent

Tools

AWS

MySQL

Flink

Kubernetes

OpenTelemetry

Kafka

Terraform

Job description

AFFIRM is seeking a Staff Software Engineer to lead the Observability effort. You’ll own the telemetry pipeline architecture, balance reliability and cost, and mentor a team of engineers while partnering with infra and product teams.

You will drive log sampling, AI-assisted observability integration, and set a year-long technical strategy to deliver high-impact features and improve incident response. Remote-first with flexible in-person options across the US.