Senior Data Platform Engineer: Scale Petabyte Pipelines

Weights & Biases; Livingston (NJ); Sur place; USD 165 000 - 242 000; Plein temps

Résumé du poste

CoreWeave is hiring a senior data engineer to design and implement scalable data systems for the integrated Weights & Biases platform. You will build batch and streaming pipelines, craft transformation frameworks, and improve data warehouse performance for large-scale analytics.

You’ll collaborate with software engineers to define event schemas, telemetry, and secure data access, while shaping metadata, governance, and semantic layers for trusted self-service analytics.

Qualifications

• Bachelor’s degree in CS/Engineering or equivalent with strong fundamentals.
• 5+ years in software or data engineering roles with production pipelines.
• Experience building production data pipelines and distributed processing systems.
• Proficiency with dbt, Dagster, Airflow or similar tools.
• Strong data modeling, SQL optimization, monitoring and testing.

Responsabilités

• Design and build scalable data systems and pipelines across production.
• Develop data transformation frameworks and enforce data quality standards.
• Improve data warehouse performance and reliability for analytics.
• Instrument product telemetry and logging for web apps and services.
• Develop metadata catalog, lineage and governance features.
• Support data-powered AI applications and on-call platform duties.

Connaissances

Python

Go

Java

Scala

SQL

Formation

Bachelor’s degree in Computer Science/Engineering or related field

Outils

dbt

Dagster

Airflow

Description du poste

CoreWeave is hiring a senior data engineer to design and implement scalable data systems for the integrated Weights & Biases platform. You will build batch and streaming pipelines, craft transformation frameworks, and improve data warehouse performance for large-scale analytics.

You’ll collaborate with software engineers to define event schemas, telemetry, and secure data access, while shaping metadata, governance, and semantic layers for trusted self-service analytics.