Senior Automation Engineer

KWAN; Porto; Vor Ort; EUR 55.000 - 75.000; Vollzeit

Zusammenfassung

KWAN in Porto seeks a Senior Automation Engineer to design and scale data automation and web scraping pipelines using Python, with a strong focus on reliability, performance, and maintainability.

You will integrate diverse data sources, including sites with anti-bot measures, build REST API integrations and event-driven workflows, apply AI-assisted extraction, monitor data quality, and collaborate with cross-functional teams to deliver robust data products.

Qualifikationen

• 5+ years in automation and data engineering, Python-first.
• Industrial-scale web scraping: Playwright or Selenium, session management, anti-bot, captchas, proxies, rate limiting.
• Speed in integrating new sources — marginal cost of adding a source defines scaling.
• API integration where available: REST, OAuth, webhooks, pagination.
• Extraction of unstructured information: PDFs, news, documents — analysis, entity extraction, normalization, deduplication.
• Public sources of statistical data (Eurostat, INE, World Bank and equivalents).
• Orchestration and reliability: scheduling, retries, idempotence, monitoring, alerting.
• Source quality: validating collections and detecting incomplete data.
• Git, code review, CI/CD and automated testing.
• Fluent level of English and Portuguese.

Aufgaben

• Design and build Python-based automation and web scraping solutions at industrial scale.
• Integrate new data sources quickly, including sites with anti-bot measures, captchas, proxies and rate limiting.
• Build and maintain integrations via REST APIs, OAuth and webhooks, including paginated data sources.
• Extract and normalize information from unstructured sources such as PDFs, reports and news articles.
• Implement orchestration, scheduling, retries, idempotency, monitoring and alerting for data pipelines.
• Validate data quality at the source and detect silent or incomplete collection failures.
• Apply LLMs and agent-based approaches for document/page extraction, news classification and summarization, and self-healing scrapers.

Kenntnisse

Python

Web scraping

Data engineering

REST APIs

CI/CD

English

Portuguese

Tools

Playwright

Selenium

Git

CI/CD

Jobbeschreibung

We are looking for a Senior Automation Engineer in Porto who's eager to embark on an exciting new opportunity!

About KWAN

We connect IT professionals with projects that match their skills, professional experience, and goals, then we offer a career progression program, delivered by in-house specialists - our People Experience Partners - to guide them through our journey with us.

What is this challenge about:

• Design and build code-based automation and web scraping solutions in Python at industrial scale.
• Integrate new data sources quickly, including sites with anti-bot measures, captchas, proxies and rate limiting.
• Build and maintain integrations via REST APIs, OAuth and webhooks, including paginated data sources.
• Extract and normalize information from unstructured sources such as PDFs, reports and news articles.
• Implement orchestration, scheduling, retries, idempotency, monitoring and alerting for data pipelines.
• Validate data quality at the source and detect silent or incomplete collection failures.
• Apply LLMs and agent-based approaches for document/page extraction, news classification and summarization, and self-healing scrapers.

What we are looking for:

• 5+ years in automation and data engineering, Python-first;
• Industrial-scale web scraping: Playwright or Selenium, session management, anti-bot, captchas, proxies, rate limiting, structure change detection;
• Speed in integrating new sources — the marginal cost of adding a source is what defines how far the team can scale;
• API integration where available: REST, OAuth, webhooks, pagination;
• Extraction of unstructured information: PDFs (reports and accounts), news and documents — analysis, entity extraction, normalization, deduplication;
• Public sources of statistical data (Eurostat, INE, World Bank and equivalents);
• Orchestration and reliability: scheduling, retries, idempotence, monitoring, alerting;
• Source quality: validating collections and detecting those that fail silently or come incomplete;
• Git, code review, CI/CD and automated testing;
• Fluent level of English and Portuguese.

What we offer

• A project that matches your skills and ambitions, as well as your preferences for working policies and culture.
• A competitive salary with awesome benefits and opportunities to leverage your knowledge and network to earn additional income.
• An empowering and respectful work culture enriched with social and learning events.
• A People Experience Partner specially assigned to you - your go-to career guide, responsible for supporting your growth, facilitating training, and ensuring your work-life balance at KWAN.

Some Words From Our Colleagues

“At KWAN, they make sure that I feel comfortable with the client I’m working for and that what I’m doing aligns with my career aspirations.”