Senior Data Engineer

Hashlist; Poissy; Sur place; EUR 85 000 - 125 000; Plein temps

Résumé du poste

Hashlist is seeking a Senior Data Engineer to design, build, and optimise robust data pipelines handling massive datasets in batch and real-time, ensuring scalable and reliable data architecture for B2B products at global scale.

You will work at the intersection of software engineering and data science, applying best practices, CI/CD with GitHub Actions, Terraform for cloud, and Spark/Scala/SQL to deliver high-quality data products across the organization in Poissy, Île-de-France.

Qualifications

• Degree in Computer Science, Engineering, Mathematics, or related field preferred.
• 5+ years in data engineering and the software development lifecycle.
• 4+ years hands-on with AWS cloud services.
• Deep expertise in Apache Spark — Core, SQL, and Structured Streaming.

Responsabilités

• Design and implement complex data processing pipelines using Apache Spark.
• Build scalable, distributed systems for high-throughput data streams and large-scale batch processing.
• Design and implement real-time pipelines with strong data quality and validation controls.
• Manage and provision cloud infrastructure using Terraform.
• Implement CI/CD workflows using GitHub Actions for automated testing and deployment.
• Uphold rigorous engineering standards — unit/integration testing, code reviews, and clean documentation.
• Translate business requirements into technical specifications alongside cross-functional stakeholders.

Connaissances

Apache Spark

Scala or Java

SQL

Python

Terraform

GitHub Actions

Kafka

Airflow

Azure Data Factory

NoSQL databases

Formation

Degree in Computer Science, Engineering, Mathematics

Outils

Terraform

GitHub Actions

Airflow

Luigi

Azure Data Factory

Description du poste

We're looking for a Senior Data Engineer to design, build, and optimise robust data pipelines that process massive datasets in both batch and real-time. Working at the intersection of software engineering and data science, you'll ensure our data architecture is scalable, reliable, and follows industry best practices - building systems that power B2B data products at global scale.

• Employment type: Permanent contract via EOR, or freelance

Responsibilities:

• Design and implement complex data processing pipelines using Apache Spark
• Build scalable, distributed systems for high-throughput data streams and large-scale batch processing
• Design and implement real-time pipelines with strong data quality and validation controls
• Manage and provision cloud infrastructure using Terraform
• Implement CI/CD workflows using GitHub Actions for automated testing and deployment
• Uphold rigorous engineering standards — unit/integration testing, code reviews, and clean documentation
• Translate business requirements into technical specifications alongside cross-functional stakeholders

Qualifications:

• Degree in Computer Science, Engineering, Mathematics, or related field preferred
• 5+ years in data engineering and the software development lifecycle
• 4+ years hands-on with AWS cloud services
• 4+ years building production data applications across relational and columnar data stores
• Deep expertise in Apache Spark — Core, SQL, and Structured Streaming
• Strong proficiency in Scala or Java for JVM-based production development
• Advanced SQL for transformation, analysis, and performance tuning
• Hands-on experience with Terraform and GitHub Actions
• Experience with Apache Flink for low-latency stream processing
• Proficiency in Python for automation and scripting
• Familiarity with Kafka, event-driven architectures, and streaming data
• Experience with workflow engines — Airflow, Luigi, or Azure Data Factory
• Familiarity with Lakehouse architectures (Delta Lake, Iceberg) or NoSQL databases