Staff Observability Engineer - Remote Backend

Affirm, Inc.; St. Louis (MO); Remote; USD 204,000 - 290,000; Full time

Benefits offered by this job

Remote-first culture

100% medical coverage for you and your

Monthly tech & wellness stipends

Flexible time off

Job summary

AFFIRM is seeking a Staff Software Engineer to lead Observability. You’ll own the architecture of the telemetry pipeline, design smart log sampling, and ensure observability is integrated into development and code reviews.

You’ll spend most of your time hands-on while guiding a six-engineer team, mentoring, and partnering with product, design and analytics to drive reliable, scalable systems across 200+ TB/day of metrics, logs, and traces.

Qualifications

• 8+ years designing, developing and launching backend systems at scale.
• Experience with Python or Kotlin.
• Distributed systems with AWS, MySQL, Flink and Kubernetes.
• Telemetry systems using OpenTelemetry, Kafka, Terraform.
• Strong observability practices and SLOs.
• Ability to drive platform standards across teams.
• Excellent communication and collaboration.
• Bachelor’s degree or equivalent practical experience.

Responsibilities

• Own the architecture of AFFIRM’s telemetry pipeline, balancing reliability, query performance, and cost at 200+ TB/day.
• Design log sampling strategies to reduce volume and cost while preserving signal.
• Shape how observability fits into agentic development and code review.
• Set technical strategy for your team on a year-long horizon and align with business goals.
• Collaborate with product, design & analytics across the development lifecycle.
• Act as a force-multiplier by defining technical solutions and operational processes.
• Ensure operations and availability with monitoring, runbooks, and on-call readiness.
• Promote code quality and design standards through writing and talks.
• Mentor and develop team members with feedback and guidance.

Skills

Backend architecture

Distributed systems

Python or Kotlin

AWS

MySQL

Kubernetes

Education

Bachelor’s degree or equivalent practical experience

Tools

AWS

MySQL

Flink

Kubernetes

OpenTelemetry

Kafka

Terraform

Job description

AFFIRM is seeking a Staff Software Engineer to lead Observability. You’ll own the architecture of the telemetry pipeline, design smart log sampling, and ensure observability is integrated into development and code reviews.

You’ll spend most of your time hands-on while guiding a six-engineer team, mentoring, and partnering with product, design and analytics to drive reliable, scalable systems across 200+ TB/day of metrics, logs, and traces.