Senior AI Infra Performance & Observability Engineer

Fdetimes; Sunnyvale (CA); On-site; USD 182,000 - 242,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Company-paid Life Insurance

Voluntary life insurance

Disability insurance

Flexible Spending Account

Health Savings Account

Tuition Reimbursement

Employee Stock Purchase Program (ESPP)

Mental Wellness Benefits

Family-Forming support

Paid Parental Leave

Job summary

CoreWeave is seeking a Senior Engineer to drive the Benchmarking & Performance team, building observability and performance insights across AI infrastructure. You will turn telemetry into real-time signals for engineers, product teams, and executives, spanning GPUs, interconnect fabrics, and distributed training workloads.

This role focuses on near real-time detection and diagnosis rather than dashboards, with opportunities to shape data lake schemas and time-series pipelines.

Qualifications

• 5+ years building distributed systems or observability tooling.
• Strong coding in Python or Go; C++ is a plus.
• Hands-on Kubernetes at production scale; CI/CD experience.

Responsibilities

• Design and build systems for continuous health and performance assessment of AI infrastructure (GPU utilization, interconnect fabrics, distributed training).
• Own time-series and metrics infrastructure; optimize PromQL/MetricsQL queries for alerting and trend analysis.
• Develop telemetry pipelines for hardware and workloads across data centers and clusters.
• Design the performance data lake with Iceberg/Parquet/Avro schemas, tiering, and schema evolution.
• Profile and tune query engines to meet strict latency/Freshness SLAs; benchmark infrastructure itself.
• Build self-service BI views (Grafana/Looker) as downstream outputs when necessary.

Job description

CoreWeave is seeking a Senior Engineer to drive the Benchmarking & Performance team, building observability and performance insights across AI infrastructure. You will turn telemetry into real-time signals for engineers, product teams, and executives, spanning GPUs, interconnect fabrics, and distributed training workloads.

This role focuses on near real-time detection and diagnosis rather than dashboards, with opportunities to shape data lake schemas and time-series pipelines.