Senior ETL Engineer — Porto (Hybrid, Data Pipelines)

Decskill; Porto; Híbrido; EUR 40 000 - 64 000; Tempo integral

Resumo da oferta

Decskill is seeking an ETL Developer for a Porto-based project with hybrid work (2x per week in the office). You will design, build, and operate ETL pipelines that import tariff and customs data via REST APIs, bulk downloads, and web crawling.

Collaborate with customs domain experts to ensure data quality and a robust relational model. The role requires 5+ years in ETL/data engineering, Python-based pipelines, strong SQL skills, and the ability to own data sources end-to-end.

Qualificações

• 5+ years of experience in ETL/Data engineering roles.
• Proven production ETL pipelines in Python (requests/httpx, BeautifulSoup, Scrapy, Playwright, pandas, SQLAlchemy).
• Experience consuming REST APIs and building web crawlers/scrapers with auth and throttling.
• Strong SQL and relational data modeling, ideally PostgreSQL.
• Ability to own data sources end-to-end from analysis to monitoring.
• Experience with XML, JSON, and CSV datasets at scale.

Responsabilidades

• Design, build, and operate ETL pipelines importing tariff and customs data via REST APIs, bulk downloads, and web scraping.
• Develop resilient web crawling logic for portals without APIs, including change detection, pagination, rate limiting, and recovery.
• Analyze source data structures per jurisdiction, including nomenclature hierarchies, duty measures, and legal references.
• Design and evolve the relational data model (PostgreSQL) balancing normalization and performance.
• Implement data quality checks, reconciliation, and data versioning for tariff changes over time.
• Monitor pipeline health and adapt pipelines to changing publication formats.
• Document data lineage, source peculiarities, and data mapping decisions.
• Collaborate with customs domain experts and engineering to align data model with requirements.

Conhecimentos

Python

SQL

Data modeling

Web crawling

Ferramentas

requests/httpx

BeautifulSoup

Scrapy

Playwright

pandas

SQLAlchemy

PostgreSQL

Descrição da oferta de emprego

Decskill is seeking an ETL Developer for a Porto-based project with hybrid work (2x per week in the office). You will design, build, and operate ETL pipelines that import tariff and customs data via REST APIs, bulk downloads, and web crawling.

Collaborate with customs domain experts to ensure data quality and a robust relational model. The role requires 5+ years in ETL/data engineering, Python-based pipelines, strong SQL skills, and the ability to own data sources end-to-end.