Site Reliability Engineer — AI Platform Observability

Appnovation; Miami (FL); On-site; USD 120,000 - 180,000; Full time

Job summary

Appnovation seeks a Site Reliability Engineer to own a shared observability platform for LLM-based applications. The role covers building monitoring, alerting, SLAs, and runbooks while managing Kubernetes on AWS and ClickHouse-based infrastructure.

You will enhance platform reliability and onboarding for internal teams. The platform currently uses Langfuse, Argo CD, Helm, and Kubernetes, with ClickHouse, PostgreSQL, and Redis as core components.

Qualifications

• Hands-on Kubernetes on AWS (EKS) operations and maintenance.
• Experience with ClickHouse replication, Keeper quorum and backup/restore.
• Experience building monitoring, alerting and service levels from scratch.
• Experience upgrading self-hosted software with rollback plans.
• PostgreSQL and Redis operations including backpressure and queue behavior.
• Strong ability to write runbooks and SOPs for on-call scenarios.

Responsibilities

• Platform Operations: diagnose failures across ClickHouse, PostgreSQL, Redis and Kubernetes.
• ClickHouse Operations: manage Keeper quorum, replication, shard/topology and S3 tiering.
• Backup/Restore: rehearse and test restores in outages and failures.
• Safe Upgrades: plan and rehearse upgrades with rollback for partial migrations.
• Monitoring/SLAs: build monitoring, alerting and SLAs from scratch.
• IaC: maintain Kubernetes manifests, Helm values, Argo CD apps.
• Runbooks/SOPs: write and maintain incident response documentation.
• Onboarding/Support: enable internal teams relying on platform.
• Automation: convert recurring tasks into automation.
• Migration partnerships: coordinate with platform owner on adoption and rollback.

Skills

Kubernetes

AWS EKS

Argo CD

Helm

ClickHouse

PostgreSQL

Redis

Observability

Langfuse

Grafana

Tools

Langfuse

LangSmith

Arize Phoenix

Job description

Appnovation seeks a Site Reliability Engineer to own a shared observability platform for LLM-based applications. The role covers building monitoring, alerting, SLAs, and runbooks while managing Kubernetes on AWS and ClickHouse-based infrastructure.

You will enhance platform reliability and onboarding for internal teams. The platform currently uses Langfuse, Argo CD, Helm, and Kubernetes, with ClickHouse, PostgreSQL, and Redis as core components.