Python Scraping Engineer - AWS & Playwright

iCapital Network, Inc.; Lisboa; Híbrido; EUR 60 000 - 90 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Hybrid work (4 days onsite, 1 day远)

Resumo da oferta

iCapital Network, Inc. is seeking a seasoned Python developer to architect and implement secure scraping solutions against complex web portals.

You will design scalable pipelines using Playwright, AWS Lambda, Step Functions, S3, and related services. You will collaborate across teams to manage workflows, ensure fault tolerance, and enable data extraction at scale, while contributing to best practices and tooling for robust data pipelines.

Qualificações

• 5+ years of Python development focused on automation and data extraction.
• Proficient in web scraping using Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
• Experience deploying scraping workloads in AWS (Lambda, S3, IAM, CloudWatch, Secrets Manager).
• Comfortable integrating APIs, JSON/XML, and performing data transformations.

Responsabilidades

• Assist with design and implementation of advanced scraping solutions for complex portals.
• Architect scraping pipelines using serverless AWS components (Lambda, Step Functions, S3, CloudWatch, Secrets Manager).
• Build scalable, fault-tolerant systems with retries and robust logging.
• Integrate workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.

Conhecimentos

Python development

Web scraping

Formação académica

Bachelor's or Master's in CS/SE or related field

Ferramentas

Playwright

Selenium

Scrapy

BeautifulSoup

requests

AWS

Lambda

S3

IAM

CloudWatch

Secrets Manager

CloudFormation

Terraform

Descrição da oferta de emprego

iCapital Network, Inc. is seeking a seasoned Python developer to architect and implement secure scraping solutions against complex web portals.

You will design scalable pipelines using Playwright, AWS Lambda, Step Functions, S3, and related services. You will collaborate across teams to manage workflows, ensure fault tolerance, and enable data extraction at scale, while contributing to best practices and tooling for robust data pipelines.