Senior Data Engineer: Pipelines, Lake Storage & Data Quality

Amo; Paris; On-site; EUR 70,000 - 110,000; Full time

Benefits offered by this job

Health insurance 100%

Generous PTO (8-9 weeks)

Parental leave (Maternity/Paternity/2P

Office in central Paris

Job summary

United States Digital Space LLC is seeking a Senior Data Engineer to design and operate batch and streaming data pipelines in Paris. The role focuses on moving production data to archival storage, building reusable datasets, and optimizing queries for cost and latency.

You will work with PySpark, Spark SQL, and Iceberg within a data-driven environment. The position is onsite in Paris, France, with a strong emphasis on data quality, observability, and scalable architectures.

Qualifications

• Senior-level experience building and operating data pipelines.
• Strong Python and SQL.

Responsibilities

• Build and maintain batch and streaming data pipelines.
• Move production data into archival/data lake storage safely and observably.
• Improve quality, reliability, and observability of derived datasets.
• Design validations for freshness, completeness, schema correctness, and data quality.
• Build reusable datasets that reduce one-off data extraction and aggregation work.
• Optimize queries and data processing jobs for correctness, latency, and cost.
• Prototype in the right tools, then integrate scoped production experiments with the company’s systems where needed.

Skills

Senior data pipelines

Python

SQL

PySpark

Spark SQL

Spark Streaming

Parquet

Iceberg

Data lakes

Data quality

Query optimization

Distributed systems

Tools

Flink

Rust

Job description

What You’ll Do

• Build and maintain batch and streaming data pipelines.

• Move production data into archival/data lake storage safely and observably.

• Improve quality, reliability, and observability of derived datasets.

• Design validations for freshness, completeness, schema correctness, and data quality.

• Build reusable datasets that reduce one-off data extraction and aggregation work.

• Optimize queries and data processing jobs for correctness, latency, and cost.

• Prototype in the right tools, then integrate scoped production experiments with the company’s systems where needed.

Must Have

• Senior-level experience building and operating data pipelines.

• Strong Python and SQL.

• Production experience with Python or Scala.

• Experience with PySpark, Spark SQL, Spark Streaming, Parquet, and Iceberg.

• Strong understanding of batch and stream processing.

• Query optimization experience.

• Familiarity with data engine internals and distributed data systems.

• Experience with data lakes, table/storage formats, and derived datasets.

Nice To Have• Experience with Flink or similar stream processing systems.

• Experience with ML training or inference pipelines.

• Familiarity with ranking, recommendations, embeddings, or entity resolution.

• Experience with privacy-sensitive data systems, GDPR workflows, or data deletion/export pipelines.

• Experience reading or making scoped changes in Rust-backed systems.

Life at the companyTo ensure that everyone is set up for success within our way of working, we work together onsite 5 days a week.

We wanted to make sure coming to the office was as comfortable as possible for you:

• We chose a location in central Paris, near Opera (Metro lines 3,8,9 and RER A).

• We have a beautiful Parisian-style office with high ceilings, balconies, and huge windows. So a lot of natural light!

Because life outside of work should also be stress-free, we cover:

• Health care (100% coverage).

• Maternity Leave, Paternity Leave, Second Parent Leave (salary maintained at 100%).

• Vacation days: 8-9 weeks (total) per year - (European summer? 🥹😎).

• 5 weeks of paid time off (by the state);

• 1-2 additional weeks (RTT, based on our company contract);

• ~11 bank holidays.

We shut down entirely twice a year, two weeks in summer and one week in winter to allow everyone to truly recharge and to avoid prolonged slowdowns (especially in the summer). These pauses are part of your total vacation time, giving everyone a real chance to unplug and recharge. No Slack, no email, no FOMO.

Find more English Speaking Jobs in France on Arbeitnow