On-Prem Data Engineer: Offline-First Pipelines

MARSS; Nice; On-site; EUR 70,000 - 110,000; Full time

Job summary

Responsibilities include ingesting, processing, transforming, storing, and making diverse data sources available to AI and software teams, while collaborating with MLOps, DevOps, and software engineers to ensure reliability and scalability. Strong Python and SQL skills are required, along with Linux proficiency and a track record of handling complex datasets.

Qualifications

• Strong professional experience in a data engineering role.
• Experience designing and implementing data pipelines from zero (not cloud-based).
• Experience with multiple data types including images, video and time-series is a plus.

Responsibilities

• Design and build data pipelines from scratch, from ingestion to consumption.
• Ingest data from offline or intermittently connected nodes with buffering and reconciliation.
• Define data prioritisation with ML team during bandwidth windows.
• Handle multiple data types: structured data, images, video, time-series.
• Develop robust data processing with Python and SQL.
• Architect data models and storage with capacity planning for large media data.
• Own workflow orchestration definitions and retry/backfill behaviour.
• Document pipeline architecture and data flows.

Skills

Python

SQL

Linux

Git

Data pipelines

Offline-first

Job description

Responsibilities include ingesting, processing, transforming, storing, and making diverse data sources available to AI and software teams, while collaborating with MLOps, DevOps, and software engineers to ensure reliability and scalability. Strong Python and SQL skills are required, along with Linux proficiency and a track record of handling complex datasets.