Data Engineer: Build Multimodal, Training-Ready Datasets

Harmattan AI; Paris; Sur place; EUR 70 000 - 95 000; Plein temps

Résumé du poste

Harmattan AI is seeking a Data Engineer in Paris to own the data layer feeding ML models, from raw logs to curated, versioned datasets. You will build training-ready datasets, manage terabytes of unstructured data, and enable modelers to focus on model development, not data wrangling.

You will collaborate with data gathering and labeling teams to ensure robust dataset construction, versioning, and efficient delivery for deep-learning pipelines in a fast-paced defense context.

Qualifications

• Educational Background: A STEM degree or equivalent practical experience.
• Data Pipelines at Scale: Built pipelines for unstructured data at scale (video, images, or sensor data).
• Engineering: Strong in Python and data engineering, and comfortable optimizing data loaders for training frameworks (e.g., PyTorch).
• Bonus: Multimodal sensor data, labeling or dataset construction for ML, dataset versioning tooling, and distributed data processing tooling.
• Attributes: Systematic, quality-minded, pragmatic, and service-oriented so the modelers are enabled, with a knack for taming messy data via automation.
• Commitment: 100% dedication to Harmattan AI's mission of providing a defensive edge to allied nations through ethical, high-impact technology.

Responsabilités

• Ingestion Pipeline: Ingest, decode, and store raw data into efficient formats.
• Multimodal Alignment: Align multiple data streams temporally and spatially for training.
• Curation: Transfer ingested data and datasets into high-value data; parsing, filtering, de-duplication and revision.
• Data & Labeling Requirements: Define data to gather and labeling workflow; coordinate with data teams.
• Dataset Construction: Build task-specific datasets for training and evaluation with cross-team collaboration.
• Versioning & Lineage: Version datasets and maintain lineage for reproducibility.
• Storage & Formats: Store data in training-ready formats and manage storage costs.
• Efficient Delivery: Deliver clean datasets and tools for loading to avoid I/O bottlenecks.

Connaissances

Python

Data engineering

Unstructured data handling

PyTorch

Dataset versioning

Formation

STEM degree or equivalent

Outils

PyTorch

ETL tooling

Description du poste

Harmattan AI is seeking a Data Engineer in Paris to own the data layer feeding ML models, from raw logs to curated, versioned datasets. You will build training-ready datasets, manage terabytes of unstructured data, and enable modelers to focus on model development, not data wrangling.

You will collaborate with data gathering and labeling teams to ensure robust dataset construction, versioning, and efficient delivery for deep-learning pipelines in a fast-paced defense context.