Senior Scraping Engineer - Build Scalable Data Pipelines

NielsenIQ; Portugal; Presencial; EUR 70 000 - 110 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Flexible working environment

Volunteer time off

LinkedIn Learning

Employee-Assistance-Program (EAP)

Resumo da oferta

NIQ Digital shelf seeks a seasoned Python developer to design and build scalable scraping systems. You will reverse engineer APIs and mobile apps, bypass anti-bot defenses, and manage proxies to deliver high-quality data at scale. Collaborate with a global team of engineers, analysts, and data lovers to pioneer innovative scraping solutions.

Your work will blend hacking mindset with engineering rigor, driving continuous improvement in data extraction, reliability, and performance.

Qualificações

• 5+ years of experience with Python and large-scale scraping projects.
• Deep understanding of anti-bot technologies and how to bypass them, fingerprinting, WAFs, JavaScript challenges.
• Experience with Scrapy, Requests, httpx, Selenium, Playwright, or custom headless browser drivers.
• Skilled at handling proxies (residential, datacenter, rotating), session management, cookie injection, and TLS tweaking.
• Strong debugging skills across browser, HTTP traffic, and device-level interactions.

Responsabilidades

• Design and build distributed scraping architectures that can scale across thousands of domains.
• Reverse engineer APIs and mobile apps using Frida, mitmproxy, Charles Proxy, Burp Suite, and many more.
• Detect common anti-bot measures to build crawlers that appear more human-like.
• Lead technical innovation in scraping: identify edge cases, prototype new approaches, and share knowledge with the team.
• Set up anomaly detection and monitoring to detect blockers or failures early.

Conhecimentos

Python

Anti-bot technologies

Scrapy

Requests

httpx

Selenium

Playwright

Proxies management

Debugging

Linux

Ferramentas

Frida

mitmproxy

Charles Proxy

Burp Suite

Descrição da oferta de emprego

NIQ Digital shelf seeks a seasoned Python developer to design and build scalable scraping systems. You will reverse engineer APIs and mobile apps, bypass anti-bot defenses, and manage proxies to deliver high-quality data at scale. Collaborate with a global team of engineers, analysts, and data lovers to pioneer innovative scraping solutions.

Your work will blend hacking mindset with engineering rigor, driving continuous improvement in data extraction, reliability, and performance.