Python Developer

Icapitalnetwork; Lisboa; Híbrido; EUR 55 000 - 85 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health insurance

Equity

Paid time off

Resumo da oferta

iCapital Network seeks an experienced Python Developer to build robust scraping solutions and automated portal interactions. You will leverage Playwright for browser automation, MFA handling, and cloud-native deployment on AWS to scale data extraction across secured portals.

You will design pipelines with Lambda, Step Functions, S3, CloudWatch, and Secrets Manager, ensuring fault tolerance, retries, and comprehensive logging. Collaboration across portals and APIs is essential.

Qualificações

• 5+ years of Python development with automation and data extraction focus.
• Experience with web scraping using Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
• Expertise in multi-step authentication, MFA, CAPTCHA solving, and session management.
• Experience deploying scraping workloads on AWS (Lambda, S3, IAM, CloudWatch, Secrets Manager).
• Knowledge of asynchronous programming, headless browsers, and JS-rendered content.
• Strong understanding of HTTP/HTTPS, cookies, headers, and API integrations.

Responsabilidades

• Design and implement advanced scraping solutions with Python, Playwright, and AWS services.
• Automate interactions with JS-heavy and authenticated sites including MFA flows.
• Architect serverless scraping pipelines using Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
• Build scalable systems with fault tolerance, retries, and robust logging.
• Coordinate workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.

Conhecimentos

Automation

Data extraction

Python programming

Asynchronous programming

Headless browsers

Formação académica

Bachelor’s or Master’s degree in Computer Science / Software Engineering

Ferramentas

Python

Playwright

Selenium

Scrapy

BeautifulSoup

requests

AWS Lambda

S3

IAM

CloudWatch

Secrets Manager

Serverless architectures

Descrição da oferta de emprego

About the Role:

We are looking for an experienced Python Developer with expertise in web scraping, automated portal interactions, and cloud-native deployment using AWS. The ideal candidate will have hands-on experience working with Playwright for browser automation, managing multi-factor authentication (MFA) flows, and deploying scalable scraping tasks via AWS Lambda and related services.

You will assist in architecting and building robust, secure, and scalable scraping solutions that interact with complex web applications and secured portals.

Responsibilities:

• Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
• Automate interactions with JavaScript-heavy and authentication-secured websites, including handling MFA, CAPTCHAs, and session/token-based login flows.
• Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
• Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
• Integrate and manage complex workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.

Qualifications:

• 5+ years of experience in Python development, with a strong focus on automation and data extraction.
• Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
• In-depth experience handling multi-step authentication flows, including multi-factor authentication (MFA), CAPTCHA solving, and session/cookie management.
• Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager.
• Experience with asynchronous programming, headless browsers, and JavaScript-rendered content.
• Solid understanding of web protocols (HTTP, HTTPS, cookies, headers), and ability to reverse-engineer network calls and authentication mechanisms.
• Comfortable working with APIs, JSON/XML, and data transformation.

Preferred Qualifications:

• Experience with CI/CD pipelines, Docker, and infrastructure as code (e.g., CloudFormation, Terraform).
• Familiarity with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions.
• Prior experience scraping from secure or enterprise-level portals (e.g., financial, healthcare, legal).
• Background in data engineering or ETL workflows is a plus.
• Exposure to Python testing frameworks and writing unit/integration tests.

Education:

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field, or equivalent professional experience.

We believe the best ideas and innovation happen when we are together. Employees in this role will work in the office four days, with the flexibility to work remotely one day (Friday).

Benefits

iCapital offers a comprehensive benefits package that includes a total compensation program consisting of competitive salary, annual performance bonus, and equity for all full-time employees; healthcare with 100% employer-paid health and dental insurance; and generous paid time off (PTO).

For additional information on iCapital Network, please visit https://www.icapitalnetwork.com/about-us Twitter: @icapitalnetwork | LinkedIn: https://www.linkedin.com/company/icapital-network-inc