Senior Scraping Engineer: Scalable Crawling & Anti-Bot

NielsenIQ; Madrid; Presencial; EUR 70.000 - 100.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Flexible hours

Volunteer time off

LinkedIn Learning

EAP

Descripción de la vacante

NIQ Digital shelf is seeking a skilled Scraping Architect to design and build distributed scraping systems that scale across thousands of domains. You will reverse engineer APIs, bypass anti-bot measures, and optimize proxies, TLS, and headless browsers.

You will lead technical innovation and share knowledge with the team. You will set up anomaly detection and monitoring to detect blockers early and collaborate with data engineers to ensure robust data pipelines.

Formación

• 5+ years of experience with Python and large-scale scraping projects.
• Deep understanding of anti-bot technologies and bypass strategies.
• Experience with Scrapy, Requests, httpx, Selenium, Playwright, or headless drivers.

Responsabilidades

• Design and build distributed scraping architectures that scale across thousands of domains.
• Detect and bypass anti-bot measures to enable human-like crawlers.
• Lead technical innovation, prototype new approaches, and share knowledge with the team.
• Set up anomaly detection and monitoring to catch blockers early.

Conocimientos

Python

Anti-bot technologies

Debugging skills

Large-scale scraping

Herramientas

Frida

mitmproxy

Charles Proxy

Burp Suite

Selenium

Playwright

Requests

httpx

Scrapy

Parquet

Avro

MongoDB

InfluxDB

Redis

Celery

Kafka

Linux

AWS

GCP

Kubernetes

Descripción del empleo

NIQ Digital shelf is seeking a skilled Scraping Architect to design and build distributed scraping systems that scale across thousands of domains. You will reverse engineer APIs, bypass anti-bot measures, and optimize proxies, TLS, and headless browsers.

You will lead technical innovation and share knowledge with the team. You will set up anomaly detection and monitoring to detect blockers early and collaborate with data engineers to ensure robust data pipelines.