Python Developer New Lisbon, Portugal

iCapital Network, Inc.; Lisboa; Híbrido; EUR 60 000 - 90 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Hybrid work (4 days onsite, 1 day远)

Resumo da oferta

iCapital Network, Inc. is seeking a seasoned Python developer to architect and implement secure scraping solutions against complex web portals.

You will design scalable pipelines using Playwright, AWS Lambda, Step Functions, S3, and related services. You will collaborate across teams to manage workflows, ensure fault tolerance, and enable data extraction at scale, while contributing to best practices and tooling for robust data pipelines.

Qualificações

• 5+ years of Python development focused on automation and data extraction.
• Proficient in web scraping using Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
• Experience deploying scraping workloads in AWS (Lambda, S3, IAM, CloudWatch, Secrets Manager).
• Comfortable integrating APIs, JSON/XML, and performing data transformations.

Responsabilidades

• Assist with design and implementation of advanced scraping solutions for complex portals.
• Architect scraping pipelines using serverless AWS components (Lambda, Step Functions, S3, CloudWatch, Secrets Manager).
• Build scalable, fault-tolerant systems with retries and robust logging.
• Integrate workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.

Conhecimentos

Python development

Web scraping

Formação académica

Bachelor's or Master's in CS/SE or related field

Ferramentas

Playwright

Selenium

Scrapy

BeautifulSoup

requests

AWS

Lambda

S3

IAM

CloudWatch

Secrets Manager

CloudFormation

Terraform

Descrição da oferta de emprego

You will assist in architecting and building robust, secure, and scalable scraping solutions that interact with complex web applications and secured portals.

Responsibilities:

• Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
• Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
• Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
• Integrate and manage complex workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.

Qualifications:

• 5+ years of experience in Python development, with a strong focus on automation and data extraction.
• Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
• Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager.
• Comfortable working with APIs, JSON/XML, and data transformation.

Preferred Qualifications:

• Experience with CI/CD pipelines, Docker, and infrastructure as code (e.g., CloudFormation, Terraform).
• Familiarity with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions.
• Prior experience scraping from secure or enterprise-level portals (e.g., financial, healthcare, legal).
• Background in data engineering or ETL workflows is a plus.
• Exposure to Python testing frameworks and writing unit/integration tests.

Education:

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field, or equivalent professional experience.

We believe the best ideas and innovation happen when we are together. Employees in this role will work in the office four days, with the flexibility to work remotely one day (Friday).

iCapital offers a comprehensive benefits package that includes a total compensation program consisting of competitive salary, annual performance bonus, and equity for all full-time employees; healthcare with 100% employer-paid health and dental insurance; and generous paid time off (PTO).