Data Engineer: Scalable AI Data Pipelines

Nous Research; Oregon; Hybride; USD 140 000 - 190 000; Plein temps

Résumé du poste

Nous Research in the United States is seeking a data engineer to own the data pipelines behind analytics and model training. You will turn multi-step, tool-conditioned interactions into reliable datasets the company can trust, across time zones with meaningful overlap to US hours.

You will build scalable ETL/ELT pipelines, ensure data quality, and collaborate with teams to enable data-driven decisions. The role focuses on long-horizon agentic data and requires strong SQL, Python, and cloud

Qualifications

• 4+ years of data engineering experience building and operating production data pipelines.
• Strong fundamentals in SQL, data modeling, and practical schema design and evolution.
• Deep understanding of storage formats and database types, and when each one fits.
• A working model of the data stack: ingestion, storage, transformation, orchestration, cataloging, and serving.
• Strong Python, plus shell skills for batch and storage operations.
• Hands-on cloud experience with object storage, batch compute, and IAM.
• Production experience with a workflow orchestration system (Airflow, Dagster, or Prefect).
• An instinct for data quality: you ask 'is the data good?' beyond just 'did the pipeline run?'.
• Extensive use of AI-assisted development tools with disciplined review.

Responsabilités

• Build, operate, and continuously improve the data pipelines that feed analytics and model training.
• Deliver reliable internal analytics so teams across Nous can make decisions on their own data.
• Wrangle long-horizon agentic data into structured, well-documented datasets for analysis and training.
• Choose ETL/ELT approaches based on destination capabilities, reproducibility, latency, and cost, and explain tradeoffs.
• Own pipeline operations in production, including failures, re-runs, backfills, and debugging across jobs.
• Design and evolve schemas, storage formats, and data models as needs change.
• Build memory-efficient processing that scales with streaming and partitioning; avoid single-machine bottlenecks.
• Communicate data location, lineage, quality, limitations, and fitness for use to engineers and non-engineers.

Connaissances

SQL

Data modeling

Python

Shell scripting

Data quality mindset

Outils

Airflow

Dagster

Prefect

Description du poste

Nous Research in the United States is seeking a data engineer to own the data pipelines behind analytics and model training. You will turn multi-step, tool-conditioned interactions into reliable datasets the company can trust, across time zones with meaningful overlap to US hours.

You will build scalable ETL/ELT pipelines, ensure data quality, and collaborate with teams to enable data-driven decisions. The role focuses on long-horizon agentic data and requires strong SQL, Python, and cloud