Senior Data Platform Engineer for AI Pipelines

Weights & Biases; Sunnyvale (CA); In loco; USD 165.000 - 242.000; Tempo pieno

Vantaggi offerti da questo lavoro

Medical, dental, vision insurance

401(k) with match

Flexible PTO

Catered lunch daily

Parental leave

Descrizione del lavoro

CoreWeave is seeking a Senior Data Engineer to design and operate scalable data systems for AI workflows within the Weights & Biases integration. You will build and optimize data pipelines, instrumentation, and data models across production systems to enable reliable analytics for AI applications.

You will collaborate across software, data science, and platform teams, applying dbt, Dagster, and Airflow to maintain data quality, performance, and observability at multi-petabyte scale.

Competenze

• Bachelor’s degree or foreign equivalent in Computer Science, Computer Engineering, Data Science, Information Systems, or a closely related field.
• At least five years of progressive professional experience in software engineering, data engineering, data infrastructure, or closely related occupation.
• Experience designing and implementing production data pipelines and distributed data-processing systems.
• Experience with data transformation or workflow-orchestration technologies such as dbt, Dagster, Airflow, or comparable platforms.
• Experience designing data models, schemas, and storage strategies for large-scale analytical databases or cloud data warehouses.
• Programming experience in Python, Go, Java, Scala, or other languages used for production data systems.
• Experience with SQL, database performance optimization, data-quality testing, and production monitoring.
• Knowledge of software-engineering practices (version control, testing, CI/CD, docs).
• Knowledge of web-application telemetry, event instrumentation, logging systems, APIs, and distributed-service architecture.
• Ability to communicate technical designs, risks, and tradeoffs to diverse stakeholders.

Mansioni

• Design and implement scalable data systems and pipelines across production systems.
• Develop data transformation and orchestration frameworks using dbt, Dagster, Airflow or comparable tools.
• Improve data warehouse performance, schemas, and storage for large-scale analytics.
• Build product telemetry and data instrumentation for web apps and backend services.
• Develop data catalog and semantic-layer capabilities for governance and self-service analytics.
• Support data-powered AI applications and platform foundations for data discovery and analytics.
• Operate and support the platform, monitor production systems, and participate in on-call rotations.

Conoscenze

Python

Go/Java/Scala

SQL

Data pipelines

Distributed systems

Airflow

Dagster

dbt

Formazione

Bachelor's degree in Computer Science

Strumenti

dbt

Dagster

Airflow

Descrizione del lavoro

CoreWeave is seeking a Senior Data Engineer to design and operate scalable data systems for AI workflows within the Weights & Biases integration. You will build and optimize data pipelines, instrumentation, and data models across production systems to enable reliable analytics for AI applications.

You will collaborate across software, data science, and platform teams, applying dbt, Dagster, and Airflow to maintain data quality, performance, and observability at multi-petabyte scale.