Staff Crawling Engineer (m/w/d)

Smile AI GmbH; München; Remote; EUR 90,000 - 120,000; Full time

Benefits offered by this job

Flexible Arbeitsgestaltung: Remote-Arb

Flexible Arbeitszeiten

Private Krankenversicherung

Weiterbildungsbudget: 1000 € pro Jahr

35 Tage Jahresurlaub

Bezahlter Parkplatz

Büroküche

Hundebegleitung am Arbeitsplatz

Job summary

Kaske Group sucht im Rahmen von Smile BI Devs eine erfahrene Staff Crawling Engineer zur end-to-end-Verantwortung von Crawling-Lösungen. Sie entwickeln robuste, produktionstaugliche Systeme für statische, dynamische und JS-gerenderte Webseiten und arbeiten eng mit Data-Pipelines zusammen.

Sie bringen langjährige Node.js/JavaScript-Expertise mit, beherrschen Browser-Automatisierung und arbeiten selbstständig an Architektur, Tests und Dokumentation.

Qualifications

• Mehr als 10 Jahre Berufserfahrung in Software-Engineering oder gleichwertige Staff-Ebene.

Responsibilities

• End-to-End-Verantwortung für zuverlässige Crawling-Lösungen übernehmen.
• Crawling-Prozesse mit geeigneter Parallelität, Scheduling und Retry-Logik entwickeln.
• Crawling-Services mit Kafka und Downstream-Datenverarbeitung integrieren.
• Vollständigkeit, Konsistenz, Aktualität und Qualität der Daten sicherstellen.
• Produktionsausfälle, Website-Änderungen und Leistungsprobleme analysieren und beheben.
• Realistische Schätzungen abgeben und Abhängigkeiten transparent kommunizieren.
• Zur Architektur beitragen, Code-Reviews unterstützen und Testbarkeit verbessern.
• Zusammenarbeit mit Engineering-, Produkt- und Business-Teams; Wissen teilen und Mentoring ohne Führungsverantwortung.

Skills

Fortgeschrittene Node.js/JavaScript

Produktions-Web-Crawlern

HTTP/HTML/DOM-Kenntnisse

Browser-Automatisierung (Puppeteer/6)

Systemresilienz & Fehlerbehandlung

Projekttiming & Schätzungen

Englisch (Arbeitsprofi)

Tools

Node.js

JavaScript

Puppeteer

Playwright

Selenium

SQL

Git

Linux

Docker

Kafka

PostgreSQL

Job description

Über uns:

The Kaske Group, headquartered in Munich, provides data, software and AI solutions for companies in the healthcare and life sciences sectors. Since 1997, we have grown into an integrated platform provider, combining one of the largest digital healthcare databases in the DACH region with data-driven software, AI and consulting services. With more than 100 customers and projects across 22 European countries, we support our clients from strategic data analysis through to implementation.

This role sits within Smile BI Devs, the Kaske Group’s software and analytics unit, where we develop data-driven products for the healthcare and life sciences industry.

To strengthen the reliability and scalability of our crawling systems, we are looking for an experienced engineer who will expand our capacity, improve technical resilience and help prepare our platform for further international growth.

Mitarbeit bei der Kaske Group bedeutet:

As our Staff Crawling Engineer, you will take end-to-end technical ownership of reliable, production-ready crawling solutions - from analysis and estimation through to deployment, monitoring and maintenance.

You will:

• Design, build and maintain Node.js-based crawling and data-collection services for static, dynamic and JavaScript-rendered websites.
• Develop reliable crawling processes with appropriate concurrency, scheduling, retries, monitoring and failure recovery.
• Integrate crawling services with Kafka and downstream data-processing systems.
• Ensure the completeness, consistency, timeliness and quality of collected data.
• Diagnose and resolve production failures, website changes and performance issues.
• Provide realistic estimates and communicate assumptions, dependencies and delivery risks early.
• Contribute to architecture, code reviews and technical standards while improving testability, documentation and maintainability.
• Collaborate across engineering, product and business teams, share knowledge and mentor other engineers without formal people-management responsibility.

Das bist Du:

• 10 years or more of professional software-engineering experience, or equivalent demonstrated Staff-level capability.
• Substantial experience building and operating production web crawlers, web-scraping systems or comparable data-acquisition services.
• Advanced Node.js and JavaScript skills, including asynchronous programming, concurrency and event-driven processing.
• Strong knowledge of HTTP, HTML, DOM structures, APIs and dynamically rendered websites.
• Practical experience with browser-automation tools such as Puppeteer, Playwright or Selenium.
• Experience designing reliable production processes involving queues, retries, rate limits, timeouts and failure recovery.
• Experience with SQL, relational databases, Git, Linux and Docker.
• Strong technical ownership, structured problem-solving and the ability to provide realistic estimates.
• Clear, proactive communication around assumptions, blockers, risks and changing timelines.
• Professional working proficiency in English.

Nice to have:

• TypeScript or Python experience.
• Experience with Kafka and PostgreSQL.
• Knowledge of CI/CD, monitoring, alerting and observability.
• Experience with proxy management, access restrictions or resilient browser automation.
• Experience working with international websites, multilingual data or e-commerce datasets.

Das bieten wir dir:

• Flexible working setup: Work remotely, with occasional office days and a workspace available whenever you need it.
• Flexible working hours: Organise your day in a way that supports both effective collaboration and focused work.
• Private healthcare insurance: Additional support for your health and well-being.
• Training budget: €1,000 per year for your professional development.
• 35 days of annual leave: Because proper rest matters.
• Paid parking: Available for added convenience when you work from the office.
• Office kitchen: Snacks, tea and coffee whenever you are onsite.
• Dog-friendly workplace: Well-behaved four-legged colleagues are welcome.