Senior Python Web Scraping Engineer - AWS & Playwright

Icapitalnetwork; Lisboa; Híbrido; EUR 55 000 - 85 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health insurance

Equity

Paid time off

Resumo da oferta

iCapital Network seeks an experienced Python Developer to build robust scraping solutions and automated portal interactions. You will leverage Playwright for browser automation, MFA handling, and cloud-native deployment on AWS to scale data extraction across secured portals.

You will design pipelines with Lambda, Step Functions, S3, CloudWatch, and Secrets Manager, ensuring fault tolerance, retries, and comprehensive logging. Collaboration across portals and APIs is essential.

Qualificações

• 5+ years of Python development with automation and data extraction focus.
• Experience with web scraping using Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
• Expertise in multi-step authentication, MFA, CAPTCHA solving, and session management.
• Experience deploying scraping workloads on AWS (Lambda, S3, IAM, CloudWatch, Secrets Manager).
• Knowledge of asynchronous programming, headless browsers, and JS-rendered content.
• Strong understanding of HTTP/HTTPS, cookies, headers, and API integrations.

Responsabilidades

• Design and implement advanced scraping solutions with Python, Playwright, and AWS services.
• Automate interactions with JS-heavy and authenticated sites including MFA flows.
• Architect serverless scraping pipelines using Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
• Build scalable systems with fault tolerance, retries, and robust logging.
• Coordinate workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.

Conhecimentos

Automation

Data extraction

Python programming

Asynchronous programming

Headless browsers

Formação académica

Bachelor’s or Master’s degree in Computer Science / Software Engineering

Ferramentas

Python

Playwright

Selenium

Scrapy

BeautifulSoup

requests

AWS Lambda

S3

IAM

CloudWatch

Secrets Manager

Serverless architectures

Descrição da oferta de emprego

iCapital Network seeks an experienced Python Developer to build robust scraping solutions and automated portal interactions. You will leverage Playwright for browser automation, MFA handling, and cloud-native deployment on AWS to scale data extraction across secured portals.

You will design pipelines with Lambda, Step Functions, S3, CloudWatch, and Secrets Manager, ensuring fault tolerance, retries, and comprehensive logging. Collaboration across portals and APIs is essential.