Foundational Data Engineer - High-Volume Video Pipelines

Harmattan AI; Paris; Sur place; EUR 70 000 - 90 000; Plein temps

Résumé du poste

A next-generation defense technology company based in Paris is seeking a Data Engineer to build and manage high-performance data infrastructures. The role involves creating pipelines for unstructured video data, optimizing GPU utilization, and ensuring data quality for ML applications. Ideal candidates should have 5-6+ years of experience with large datasets and strong skills in distributed computing. Join us to contribute to innovative solutions in autonomous defense systems.

Qualifications

• 5-6+ years of experience with unstructured data processing.
• Deep knowledge of operating systems, networking, and parallel computing.
• Strong command of ML data versioning tools.

Responsabilités

• Build ETL/ELT pipelines for raw video data.
• Develop algorithms to synchronize EO and IR frames.
• Architect storage-to-GPU pipelines to prevent I/O bottlenecks.
• Write distributed data processing jobs for tactical video logs.
• Implement automated quality checks and versioning.
• Evaluate storage solutions to manage growing datasets.

Connaissances

Building and maintaining terabyte-scale pipelines

Maximizing multi-node GPU utilization

Distributed computing tools (Spark, Ray, Beam)

Automating data quality checks

Formation

BS or MS in Computer Science, Software Engineering, or Distributed Systems

Outils

Apache Spark

Ray

Apache Beam

PyTorch

JAX

DVC

Apache Iceberg

Pachyderm

Description du poste

A next-generation defense technology company based in Paris is seeking a Data Engineer to build and manage high-performance data infrastructures. The role involves creating pipelines for unstructured video data, optimizing GPU utilization, and ensuring data quality for ML applications. Ideal candidates should have 5-6+ years of experience with large datasets and strong skills in distributed computing. Join us to contribute to innovative solutions in autonomous defense systems.