Full-Stack Engineer | Web Crawling (m/f/d)

Certivity; München; Hybride; EUR 90 000 - 120 000; Plein temps

Avantages offerts par ce poste

60% remote option

30 vacation days including company休? (

Résumé du poste

Certivity GmbH in Munich, Germany is seeking a software engineer to own crawlers end-to-end, from discovery to resilient fetching and turning HTML/PDF into structured data. You will work on large-scale sites, fight anti-bot defenses, and design pipelines for ingestion, deduplication, and enrichment.

The role emphasizes Python development, Docker, and cloud deployments (GCP/AWS/Azure) with CI/CD, in a small, fast-moving team. Remote work up to 60% is available in Munich’s vibrant office culture.

Qualifications

• .
• 3+ years of professional software development (5+ for the senior track).
• Strong Python - real production code, async.
• Hands-on web scraping at scale, including firsthand experience with anti-bot defenses and the failure modes of large-scale collection.
• A data-correctness mindset - you care that extracted data is right, not just that the job finished.
• Docker + at least one major cloud (GCP, Azure, or AWS), Git, comfortable on the CLI.
• Clear communicator who works well in a small, fast-moving team.

Responsabilités

• Build and scale crawlers that handle dynamic, large-scale sites - and keep them running as those sites change and fight back.
• Stay ahead of anti-bot measures: TLS/browser fingerprinting, proxies, session and rate-limit strategy, change detection.
• Turn raw HTML and PDF into clean, correct structured data - including LLM-assisted extraction - where silently-wrong output is worse than a crash.
• Design and operate pipelines for ingestion, deduplication, versioning, and enrichment.
• Ship and run containerized apps in the cloud, with CI/CD.
• Shape our technical architecture and engineering culture.

Connaissances

Python

Web scraping at scale

Data quality mindset

Clear communicator

Outils

Docker

GCP

Azure

AWS

Git

CLI

Kubernetes

Terraform

React

Description du poste

About the Job

We build and operate large-scale web crawling and extraction systems that turn unstructured, dynamic websites into clean, structured, and correct data. You'll own crawlersend-to-end: discovery, resilient fetching against real anti-bot defenses, and turningHTML/PDF into trustworthy structured output.

What you will do

• Build and scalecrawlers that handle dynamic, large-scale sites - and keep them runningas those sites change and fight back.

• Stay ahead ofanti-bot measures: TLS/browser fingerprinting, proxies, session andrate-limit strategy, change detection.
• Turn raw HTML and PDF into clean,correct structured data - including LLMassistedextraction - where silently-wrong output is worse than a crash.
• Design and operate pipelines for ingestion, deduplication, versioning, and enrichment.
• Ship and run containerized apps in the cloud, with CI/CD.
• Shape our technical architecture and engineering culture.

Our Tech StackPython, Docker, GCP (Cloud Run jobs) and Azure Blob, MongoDB. Go services and a React frontend on the periphery. You don't need all of it on day one, but you do need toramp up fast

Your profile

3+ years of professional software development (5+ for the senior track).

• Strong Python - real production code, async.
• Hands-on web scraping at scale, including firsthand experience with anti-bot defenses and the failure modes of large-scale collection.

• A data-correctness mindset - you care that extracted data is right, not just that the jobfinished.
• Docker + at least one major cloud (GCP, Azure, or AWS), Git, comfortable on theCLI.
• Clear communicator who works well in a small, fast-moving team.

Nice to have

• LLM-assisted extraction / working with LLMs, ML pipelines, or datalabeling workflows.
• Go or Java/Spring Boot.
• Frontend (React) — or eagerness to pick it up.
• Terraform, Kubernetes, modern DevOps tooling.

Why us?

• A vibrant in-office culturein Munich with the option for up to 60 % work from home.
• Flexibleworking hours.
• 30 vacation days per year (including 4 “company rest days” over christmas).
• 30 daysof "workation" per year, within the EU and selected countries.
• Highautonomyand flat hierarchies.
• EGYM Wellpassfor unlimited access to fitness courses and gyms.
• Udemy access for educational videos.

ClosingCertivity is an equal opportunities employer. We are committed to equal employment opportunities regardless of race, religion, sexual orientation, age, marital status, disability, or gender identity. Please do not submit personal data revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union membership, data concerning your health, or data concerning your sexual orientation.

Find Jobs in Germany on Arbeitnow