Data Engineer: Python, Dask & Polars - Pipelines & CI/CD

Towards AI, Inc.; Paris; Ibrido; EUR 60.000 - 90.000; Tempo pieno

Descrizione del lavoro

Towards AI, Inc. is seeking a Data Engineer to design, develop and maintain robust data pipelines in Paris. The role focuses on Python-based data processing with Polars or Dask, handling Parquet, CSV and JSON formats, and implementing automated data quality controls.

You will work in an industrial environment with containerization and orchestration, using Git and CI/CD to ensure reliable deployments and traceability. Join a team focused on scalable data solutions and security-aligned practices.

Competenze

• At least three years of data engineering experience.
• Strong Python and data ecosystem skills (Polars or Dask).
• Experience with Parquet, CSV, and JSON formats.
• Experience with containers, orchestration, Git and CI/CD practices.

Mansioni

• Design, develop and maintain robust data pipelines for large data volumes.
• Implement advanced transformation rules and data quality controls.
• Collaborate with security/compliance to identify sensitive data.
• Measure processing performance and optimize resource use for scalability.
• Ensure full traceability of code and datasets and participate in code reviews.

Conoscenze

Python

Polars

Dask

Data engineering

Strumenti

Docker

Git

CI/CD

Descrizione del lavoro

Towards AI, Inc. is seeking a Data Engineer to design, develop and maintain robust data pipelines in Paris. The role focuses on Python-based data processing with Polars or Dask, handling Parquet, CSV and JSON formats, and implementing automated data quality controls.

You will work in an industrial environment with containerization and orchestration, using Git and CI/CD to ensure reliable deployments and traceability. Join a team focused on scalable data solutions and security-aligned practices.