Senior Data Engineer

Jobgether; France; Remote; EUR 67,000 - 100,000; Full time

Benefits offered by this job

Fully remote across Europe/LatAm

Technical ownership of data platform

Modern tech stack: Python, PostgreSQL,

DBT, GCP, AI-powered extraction

End-to-end impact on product and data

Job summary

Jobgether is seeking a Senior Data Engineer based in France to own and evolve a production-grade data platform. You will design scalable pipelines ingesting APIs, webhooks, files, and web-scraped data, with an emphasis on concurrent processing and reliability.

You will implement batch and event-driven workflows, orchestrate with Airflow/ Dagster/ Prefect, and ensure data quality, lineage, and governance across raw to curated layers in GCS and PostgreSQL.

Qualifications

• End-to-End Data Engineering experience building production data platforms.
• Hands-on with Airflow, Dagster, Prefect or Temporal in production.
• Practical GCP and dbt knowledge for transformation and orchestration.
• Experience with event-driven architectures and concurrency.
• Strong Python skills for async requests and processing.
• Advanced SQL and PostgreSQL data modeling and optimization.
• Data contracts, raw/curated layers, retention policies, validation.
• Experience with web scraping pipelines and concurrency control.
• Independent ownership in a startup or lean team environment.
• Strong English communication in a distributed team.

Responsibilities

• Own data platform architecture from design to deployment and maintenance.
• Build multi-source ingestion pipelines for APIs, webhooks, files, and scraping.
• Develop batch and event-driven workflows with orchestration tools.
• Ensure reliable failure recovery with retries and dead-letter queues.
• Maintain raw-to-curated data layers in GCS and dbt transformations.
• Strengthen data quality, governance, validation and monitoring.
• Optimize PostgreSQL schemas and incremental loading for performance.
• Integrate enrichment and AI extraction into the platform.
• Maintain data traceability and allow safe reprocessing and auditing.
• Drive continuous improvement for throughput, reliability, and infra efficiency.

Skills

Data platform architecture

Python programming

SQL and PostgreSQL

Data pipelines

Workflow orchestration

Web scraping

GCP and dbt

Event-driven systems

Data quality & governance

English communication

Tools

Airflow

Dagster

Prefect

Temporal

Scrapy

Playwright

Selenium

Beautiful Soup

dbt

BigQuery

Dataform

PostgreSQL

GCP

Pub/Sub

Cloud Tasks

Job description

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Data Engineer based in France.

As a Senior Data Engineer, you will take ownership of the data platform, shaping its architecture and ensuring reliable performance from development through production. You will build scalable pipelines that transform high volumes of data from APIs, webhooks, files, and web scraping into trusted, product-ready information. A key priority will be evolving existing sequential workflows into a modern platform supporting concurrent processing, event-driven ingestion, and orchestrated batch execution. You will establish robust data quality standards, resilient recovery mechanisms, and transparent data lineage to ensure accuracy and consistency at scale. Working with technologies such as Python, PostgreSQL, dbt, and Google Cloud Platform, you will integrate external data sources and AI-powered extraction capabilities. This fully remote opportunity offers broad technical ownership, the freedom to shape foundational data infrastructure, and the chance to make a direct impact in a collaborative, fast-moving engineering environment.

Accountabilities

• Own Data Platform Architecture: Design, build, and operate a production-grade data platform from initial architecture through deployment, optimization, and ongoing maintenance.

• Build Multi-Source Ingestion Pipelines: Develop concurrent ingestion systems for provider APIs, webhooks, files, and web scraping, incorporating rate limiting, error handling, and reliable processing at scale.

• Develop Batch and Event-Driven Workflows: Implement scheduled pipelines, historical backfills, and dependency management using orchestration tools such as Airflow, Dagster, or Prefect. Build event-driven processing with messaging and task queue services, including Pub/Sub or Cloud Tasks.

• Ensure Reliable Failure Recovery: Implement retries, dead-letter queues, controlled replay mechanisms, and idempotent upserts to recover from failures without generating duplicate records or corrupting data.

• Establish Raw-to-Curated Data Architecture: Preserve original source payloads, including provider JSON, in Google Cloud Storage (GCS), then transform and standardize data using dbt. Maintain clearly defined raw, standardized, and curated data layers.

• Strengthen Data Quality and Governance: Define data contracts, automated validation rules, pipeline tests, retention policies, and monitoring systems to track data freshness, processing failures, and overall reliability.

• Optimize Database Performance: Design and maintain PostgreSQL schemas, queries, and incremental loading strategies to ensure efficient processing and dependable data delivery to downstream products.

• Integrate Enrichment and AI Extraction: Incorporate third-party enrichment providers and large language model (LLM)-based structured extraction from text into the data platform.

• Ensure Data Traceability: Maintain clear lineage between source data and curated records, enabling safe reprocessing, historical recovery, auditing, and consistent data management.

• Drive Continuous Improvement: Identify opportunities to improve throughput, reliability, maintainability, and infrastructure efficiency while adapting the platform to evolving business and product needs.

Requirements:

• End-to-End Data Engineering Experience: Proven experience building production data platforms from the ground up, including architecture, pipeline implementation, deployment, monitoring, and ongoing operations.

• Workflow Orchestration: Hands-on production experience with Airflow, Dagster, Prefect, or Temporal, including workflow dependencies, retries, scheduling, and failure recovery. Airflow experience is preferred.

• Google Cloud and dbt: Practical production experience with Google Cloud Platform (GCP) and dbt, including data transformation, pipeline management, and cloud-based data infrastructure.

• Event-Driven Architecture: Experience developing asynchronous, event-driven systems using messaging platforms or task queues, with a strong understanding of concurrent processing, retries, and fault tolerance.

• Python Programming: Strong Python skills, particularly asynchronous HTTP requests and concurrent processing using libraries such as asyncio, aiohttp, or HTTPX.

• SQL and PostgreSQL: Advanced SQL knowledge and practical PostgreSQL experience, including data modeling, query optimization, incremental loading, and performance tuning.

• Data Quality and Modeling: Experience designing data contracts, raw and curated storage layers, retention policies, automated validation, and data quality monitoring.

• Web Scraping Expertise: Hands-on experience building and maintaining production scraping pipelines using tools such as Scrapy, Playwright, Selenium, or Beautiful Soup, including concurrency management, rate limiting, and failure handling.

• Independent Ownership: Ability to make sound technical decisions, manage complex projects, and take responsibility for data infrastructure across the full development lifecycle.

• Startup or Small-Team Experience: Experience working in a startup or lean engineering team, where priorities evolve quickly and engineers take broad ownership, is a plus.

• Entity Resolution: Familiarity with matching company and individual records across multiple sources, identifying duplicates, and consolidating information into consistent entities is highly desirable.

• LLM-Based Data Processing: Experience using large language models for structured data extraction, company enrichment, or people data enrichment is an advantage.

• Additional Data Technologies: Familiarity with BigQuery and Dataform is a plus.

• Communication Skills: Ability to communicate technical concepts clearly, collaborate effectively with distributed teams, and work comfortably in an English-speaking professional environment.

Benefits:

• Fully remote work: Work remotely from eligible locations across Europe and Latin America, with flexibility to organize your working environment.

• Technical ownership: Lead the architecture and evolution of a data platform, with meaningful influence over technical decisions and engineering practices.

• Modern technology stack: Work with Python, PostgreSQL, GCP, dbt, event-driven systems, workflow orchestration, web scraping, and AI-powered data extraction.

• End-to-end impact: Build foundational infrastructure that directly supports product functionality, data quality, and business growth.

• Engineering autonomy: Take ownership of technical challenges and shape scalable solutions in a dynamic, product-focused environment.

• Innovation opportunities: Explore advanced data processing, entity resolution, external data enrichment, and LLM-based extraction techniques.

• International collaboration: Work with a distributed team across multiple regions, with opportunities to contribute to evolving technical initiatives.

• Compensation flexibility: Salary expectations are discussed in USD on a monthly basis, with the final package determined through the recruitment process.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.