Staff Data Engineer

GitGuardian; Paris; Hybride; EUR 120 000 - 170 000; Plein temps

Résumé du poste

GitGuardian is seeking a Staff engineer for the Public Intelligence squad to own the end-to-end architecture of the public data platform. You’ll write production code on critical paths, while shaping a technical strategy with the Engineering Manager and Product Manager.

You’ll lead design workshops, align multiple squads, and mentor engineers to raise the team’s technical bar. This role emphasizes real-time processing, data lakes, and AI-assisted development.

Qualifications

• 7+ years building and operating large-scale, data-intensive systems in production, as a data engineer or a backend engineer.
• Strong software engineering: designed and run high-throughput collection or ingestion services, with queues, back-pressure, retries and idempotency.
• Production experience with real-time, event-driven processing and the latency targets that come with it.
• Strong data architecture: data lakes on object storage, analytical databases at scale (ClickHouse or similar), and data modeling for pipelines.
• Python expert.
• Proven experience redesigning a production architecture step by step, migrating without downtime or data loss.
• An operations mindset: you design observability and alerting together with the system, and you know how to run an incident.
• Technical leadership: you align several engineers on a target architecture, mentor them, and explain your decisions in writing.
• Deliberate use of AI coding agents: you give them context and a spec, read what they produce, and catch what they get wrong.
• Fluent English in an international environment.

Responsabilités

• Own the end-to-end architecture of the public data platform: collection, real-time processing, storage, and delivery to our agents and product.
• Stay hands-on: write production code on the most critical parts, starting with the collectors, the real-time processing and the migrations.
• Turn the squad’s priorities into a technical strategy and roadmap, together with the Engineering Manager and the Product Manager.
• Lead design workshops, and make decisions the squad can build on.
• Drive the architecture decisions that span several squads, and get them aligned.
• Mentor the squad’s engineers and raise the technical bar through design reviews and knowledge sharing.
• Lead the squad’s software factory. AI agents write a growing share of our code, and you will define how we build with them.

Connaissances

Data-intensive systems

Real-time processing

Python expert

Data architecture

Backend design

Technical leadership

Operations mindset

AI coding agents

English fluent

Outils

RabbitMQ

Kafka

Dagster

Snowflake

ClickHouse

Django

Celery

Redis

Description du poste

About GitGuardian

GitGuardian is a global cybersecurity scale-up. The company is based in Paris, New York City, Boston.
Our early investors include Scott Chacon, co-founder of GitHub, and Solomon Hykes, co-founder of Docker, alongside leading US and European VC firms. We raised our Series C in early 2026.
GitGuardian leads the way in securing the credential layer, helping organizations protect the secrets that let code, machines, and AI agents access systems with broad detection, identity context, and remediation at scale. Already trusted by 600K+ developers worldwide and by 50 of the Fortune 500!

About your team and your mission

You will join the Public Intelligence squad, which builds the engine behind GitGuardian's mission to protect companies from secrets leaked in public. We collect public data at scale, from every public commit pushed to GitHub to public Docker Hub images, and find the secrets in it. AI agents then work out which company each leak belongs to and how critical it is. Customers see these leaks in their GitGuardian workspace, and the most critical ones help our sales team start conversations with new prospects.

We ingest about 12.5 million public GitHub commits a day and store nearly 400 million secret occurrences in ClickHouse. The systems run in production, and AI agents already analyze every public secret we find. They grew fast, one project at a time, and we now have a real job ahead of us: rethink how we collect, store and process public data, so the platform is faster, cheaper, more reliable and ready for new sources.

As the squad’s Staff engineer, you will own this architecture end to end, from the collectors that read public sources to the real-time processing that feeds our agents and our product. The role sits between software and data. Our hardest problems are as much about collecting and processing a huge, rate-limited stream reliably as about how the data is stored and modeled.

You will report to Alexis, Senior Engineering Manager, and work with a squad of 5 engineers (software, data and AI) and a Product Manager.

Your impact will go beyond the squad. Public data flows through systems owned by other squads, and the architecture you design will shape their work too:

• Secrets Detection builds the engine that finds secrets in the data we collect, and the analyzers our agents use to assess the impact of a leak.

• Incidents builds the incident experience of the GitGuardian platform. Public incidents are a shared object between our squads: we qualify them and compute their risk score, and customers handle them in the same product as every other incident.

• SRE runs the infrastructure our collectors and pipelines rely on.

• Data & AI runs the company's data platform, which our pipelines share today.

Key challenges

• Design the next generation of our collectors. The GitHub public feed is huge, rate-limited and noisy. Our collectors must keep up with it, measure how much of it they cover, absorb traffic spikes, and recover from an incident without overloading the rest of the pipeline. The same design then has to work for every new source.

• Move public data processing to real time. Part of our processing still waits for daily batches. You will design the event-driven processing that takes a commit from GitHub to a qualified incident in minutes (scanning, attribution, agent analysis), with clear latency targets and back-pressure when volume spikes.

• Build a single source of truth for public data. Move public data and its transformations out of Snowflake into an S3 + Iceberg data lake that every GitGuardian service can rely on.

• Simplify a multi-service architecture. The platform grew one project at a time: collectors, queues and workers, a Django product backend, Dagster pipelines, ClickHouse and Snowflake. You will define the target architecture and lead the migration step by step, without stopping production or losing data.

• Make the platform operable. Define what healthy means for each part of the pipeline (coverage, latency, data quality), and build the monitoring and alerting that go with it.

Your responsibilities

• Own the end-to-end architecture of the public data platform: collection, real-time processing, storage, and delivery to our agents and product.

• Stay hands-on: write production code on the most critical parts, starting with the collectors, the real-time processing and the migrations.

• Turn the squad’s priorities into a technical strategy and roadmap, together with the Engineering Manager and the Product Manager.

• Lead design workshops, and make decisions the squad can build on.

• Drive the architecture decisions that span several squads, and get them aligned.

• Mentor the squad’s engineers and raise the technical bar through design reviews and knowledge sharing.

• Lead the squad’s software factory. AI agents write a growing share of our code, and you will define how we build with them: the skills, tools and conventions that make them reliable, so the whole squad ships faster without lowering the bar.

Technical environment

• Languages: Python, SQL

• Backend and processing: Django, Celery, RabbitMQ, Redis

• Orchestration: Dagster, AWS Batch

• Storage and analytics: S3, Iceberg, ClickHouse, Snowflake, PostgreSQL

• Ingestion: Vector

• Deployment: Docker, Kubernetes, Terraform, ArgoCD, GitLab CI

• Cloud: AWS, OVH

• AI: LangGraph, LiteLLM, Claude Code

• Observability: Coralogix, Sentry, OpenTelemetry

About you

Here's what we consider essential for success in this role:

• 7+ years building and operating large-scale, data-intensive systems in production, as a data engineer or a backend engineer.

• Strong software engineering: you have designed and run high-throughput collection or ingestion services, and you know their hard parts well: queues and workers, back-pressure, retries and idempotency, deduplication, rate-limited external APIs.

• Production experience with real-time, event-driven processing (RabbitMQ, Kafka or similar) and the latency targets that come with it.

• Strong data architecture: data lakes on object storage, analytical databases at scale (ClickHouse or similar), and data modeling for pipelines.

• Python expert.

• Proven experience redesigning a production architecture step by step, migrating without downtime or data loss.

• An operations mindset: you design observability and alerting together with the system, and you know how to run an incident.

• Technical leadership: you align several engineers on a target architecture, mentor them, and explain your decisions in writing.

• Deliberate use of AI coding agents: you give them context and a spec, read what they produce, and catch what they get wrong.

• Fluent English in an international environment.

The following skills would strengthen your application but aren't required:

• Apache Iceberg or other open table formats.

• Large-scale crawling.

• AWS, Kubernetes and Terraform in production.

• LLM agents in production pipelines.

• An interest in cybersecurity and in how secrets leak.

• Experience in high-growth startups or scale-ups.

The interview process

1. Video call with a recruiter

To discover your professional project and evaluate if there could be a mutual match

2. Interview with your future manager (1h)

Purpose: To know more about you and your achievements, and to present the squad.

Skills Assessed: Ownership, communication, motivation, and your experience as a technical leader.

3. Technical interview with the team (2h)

Purpose: System design discussions on collection and real-time processing, and a hands-on exercise on public GitHub data.

Skills Assessed: Software and data architecture, distributed systems, data modeling, and how you reason about trade-offs.

4.1 Final interviews with Pierre, VP of Engineering, and Eric, Co-founder and CEO

Purpose: To know more about your ambitions, and to share our company's vision for the next couple of years.

4.2 References check

You can start thinking about two contacts who can attest to your previous or current professional experiences. These contacts should be as recent as possible, and we will call them at the end of the process.