Data Engineer: High-Throughput ML Video Pipelines

Harmattan AI; Paris; On-site; EUR 70,000 - 110,000; Full time

Job summary

Harmattan AI, a Paris-based defense technology company, seeks a Data Engineer to build data pipelines for multimodal video data. You will optimize ingestion, synchronization, and loading to accelerate ML workflows and sustain high GPU utilization in multi-node clusters.

Based in Paris, you will work on terabyte- to petabyte-scale datasets, deploy scalable storage solutions, and collaborate with ML engineers to design robust, reproducible data architectures for mission-critical models.

Qualifications

• Experience building large-scale data pipelines for multimodal data.
• Strong understanding of distributed data processing and storage systems.
• Ability to optimize data ingestion, synchronization and loading for ML workloads.

Responsibilities

• Build and optimize massive data pipelines for multimodal video data.
• Improve ingestion, synchronization, and loading to accelerate ML workflows.
• Deploy scalable storage solutions and manage terabyte to petabyte datasets.
• Collaborate with ML engineers to design robust data architectures for models.

Skills

Data pipelines

ETL pipelines

Distributed systems

Video data processing

GPU utilization

Tools

Spark

Airflow

Kubernetes

Parquet

Job description

Harmattan AI, a Paris-based defense technology company, seeks a Data Engineer to build data pipelines for multimodal video data. You will optimize ingestion, synchronization, and loading to accelerate ML workflows and sustain high GPU utilization in multi-node clusters.

Based in Paris, you will work on terabyte- to petabyte-scale datasets, deploy scalable storage solutions, and collaborate with ML engineers to design robust, reproducible data architectures for mission-critical models.