Senior Software Engineer (Observability)

Coreweave; Sunnyvale (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

CoreWeave in Sunnyvale, CA seeks a Senior Software Engineer specializing in Observability to design and operate scalable logging, metrics, and tracing platforms for AI workloads.

You will build high-throughput telemetry pipelines, improve performance and security, and participate in on-call rotations while collaborating with internal teams to embed observability as a product across systems.

Qualifications

• 5+ years in software or infrastructure engineering designing, building, and operating large-scale distributed systems.
• Experience with elastic, distributed architectures and operation of observability platforms.
• Proficient in Go or Python, with production-grade code, testing, and release discipline.
• Hands-on Kubernetes experience in production, including containerization and microservices.

Responsibilities

• Design, build, and own scalable observability infrastructure for logging, metrics, and tracing.
• Develop and operate high-throughput telemetry pipelines for platform data and migrations.
• Participate in on-call rotations to support production systems and perform root-cause analysis.
• Collaborate with internal teams to embed observability as a product for platform users.

Skills

Go

Python

Kubernetes

Helm & YAML

Terraform

OpenTelemetry

Grafana

Prometheus

Tools

Loki

ClickHouse

Elasticsearch

Prometheus

Grafana

Thanos

Kafka

Kafka Connect

Job description

• We are seeking Senior Software Engineers who specialize across the pillars of Observability to play a pivotal role in CoreWeave’s ability to bring the best performing systems to market
• Whether you focus on metrics, logging and tracing, or telemetry pipelines and visualization, you’ll have an outsized opportunity to enable both CoreWeave and its customers to understand, troubleshoot, and optimize complex systems at the forefront of Artificial Intelligence
• Design, build, and own core observability infrastructure, including highly scalable and reliable logging, metrics, and tracing platforms
• Develop and operate high-throughput telemetry pipelines that ingest, transform, and expose observability data, ensuring reliability, security, and transparent data migrations for platform consumers
• Tackle observability challenges at extreme scale, supporting clusters of thousands of GPUs, petabyte-scale telemetry, and high-cardinality workloads
• Continuously improve performance, security, reliability, and scalability of observability services through software enhancements, automation, and new feature development
• Participate in the team’s on-call rotation to support critical production systems, focusing on root cause analysis and building durable solutions to prevent recurrence
• Collaborate closely with internal engineering teams, applying a platform-as-a-product mindset to understand their needs and embed observability best practices and custom tooling into their systems
• Contribute to the overall observability strategy, influencing the direction of our platform and the experience we provide to customers

5+ years of experience in software or infrastructure engineering, with a proven track record of designing, building, and operating large-scale distributed systems in productionAbility to analyze and decompose complex problems in elastic, distributed architectures into manageable, well-scoped workComfortable working with Helm and YAML-based configuration for deploying and managing services, including templating, automation, and infrastructure-as-code practicesDemonstrated experience designing, building, and delivering robust and scalable systems, with a commitment to operational excellence, high-quality code, effective testing, and progressive release strategiesA customer-obsessed, platform-minded engineer, eager to provide infrastructure as a service and apply a product lens when evaluating platform scale problemsProficient in Go (our primary language) or Python, with the ability to write clean, resilient, and testable production codeHands-on Kubernetes experience in production, including containerization and microservices architectures, and familiarity with their observability challengesExperience participating in an on-call rotation for critical production systemsDirect, hands-on experience designing, operating, or scaling logging, tracing, and/or metrics platforms (e.g., Loki, ClickHouse, Elasticsearch, Prometheus, VictoriaMetrics, Grafana, Thanos)Familiarity with data streaming systems (e.g., Kafka, Kafka Connect) for observability pipelinesExperience automating and provisioning infrastructure as part of the software development lifecycle, using tools like TerraformExperience with OpenTelemetry for unified telemetry collection and instrumentationExperience with modern AI platforms and workloads (e.g., large-scale training and inference, GPU-based infrastructure, MLOps tooling) is a plus