Senior Data Pipeline Engineer (Batch & Streaming)

amo; Paris; Sur place; EUR 70 000 - 110 000; Plein temps

Avantages offerts par ce poste

Onsite 5 days a week

Central Paris office near Opera

Résumé du poste

amo is seeking a senior data engineer to own and operate batch and streaming data pipelines in a production environment in Paris. You will move data into archival storage, improve data quality and observability, and design validations for freshness, completeness, and schema correctness.

You will build reusable datasets and optimize queries for cost and latency. Expect strong Python/SQL skills, production experience with Python or Scala, and familiarity with PySpark, Spark SQL, Spark Streaming,

Qualifications

• Senior-level experience building and operating data pipelines.
• Strong Python and SQL.
• Production experience with Python or Scala.
• Experience with PySpark, Spark SQL, Spark Streaming, Parquet, and Iceberg.
• Strong understanding of batch and stream processing.
• Query optimization experience.
• Familiarity with data engine internals and distributed data systems.
• Experience with data lakes, table/storage formats, and derived datasets.

Responsabilités

• Build and maintain batch and streaming data pipelines.
• Move production data into archival/data lake storage safely and observably.
• Improve quality, reliability, and observability of derived datasets.
• Design validations for freshness, completeness, schema correctness, and data quality.
• Build reusable datasets that reduce one-off data extraction and aggregation work.
• Optimize queries and data processing jobs for correctness, latency, and cost.
• Prototype in the right tools, then integrate scoped production experiments with amo’s systems where needed.

Connaissances

Python

MySQL/SQL

Scala

Distributed data systems

Batch and streaming processing

Query optimization

Data lake/storage

Data pipelines

Outils

PySpark

Spark SQL

Spark Streaming

Parquet

Iceberg

Description du poste

amo is seeking a senior data engineer to own and operate batch and streaming data pipelines in a production environment in Paris. You will move data into archival storage, improve data quality and observability, and design validations for freshness, completeness, and schema correctness.

You will build reusable datasets and optimize queries for cost and latency. Expect strong Python/SQL skills, production experience with Python or Scala, and familiarity with PySpark, Spark SQL, Spark Streaming,