Senior Scraper Engineer: Scale Giant Data Pipelines

Nabla; Lisboa; Presencial; EUR 60 000 - 80 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Flexible working environment

Volunteer time off

LinkedIn Learning

Employee-Assistance-Program (EAP)

Resumo da oferta

NIQ Digital shelf is seeking a seasoned Python developer to design and build scalable scraping architectures. You will reverse engineer APIs and mobile apps, detect anti-bot measures, and craft crawlers that emulate human behavior.

You’ll lead technical innovation, share knowledge, and set up monitoring to preempt blockers. You’ll work with a global team, applying your expertise to process vast data and deliver actionable insights for leading retailers and brands.

Qualificações

• 5+ years of Python and large-scale scraping experience.
• Deep knowledge of anti-bot technologies and bypass strategies.
• Experience with Scrapy, Requests, httpx, Selenium, Playwright or custom headless drivers.
• Skilled at handling proxies (residential, datacenter, rotating), session management, cookie injection, TLS tweaking.
• Strong debugging skills across browser, HTTP traffic, and device-level interactions.

Responsabilidades

• Design and build distributed scraping architectures across thousands of domains.
• Reverse engineer APIs and mobile apps using Frida, mitmproxy, Charles Proxy, burp and more.
• Detect common anti-bot measures to build crawlers that appear more human-like.
• Lead technical innovation in scraping: identify edge cases, prototype new approaches, share knowledge with the team.
• Set up anomaly detection and monitoring to detect blockers or failures early.

Conhecimentos

Python

Anti-bot technologies

Problem solving

Debugging skills

Independent thinker

Ferramentas

Scrapy

Requests

httpx

Selenium

Playwright

Frida

mitmproxy

Charles Proxy

Burp Suite

Kubernetes

Celery

Kafka

Descrição da oferta de emprego

NIQ Digital shelf is seeking a seasoned Python developer to design and build scalable scraping architectures. You will reverse engineer APIs and mobile apps, detect anti-bot measures, and craft crawlers that emulate human behavior.

You’ll lead technical innovation, share knowledge, and set up monitoring to preempt blockers. You’ll work with a global team, applying your expertise to process vast data and deliver actionable insights for leading retailers and brands.