ETL Developer

Decskill; Porto; Híbrido; EUR 40 000 - 64 000; Tempo integral

Resumo da oferta

Decskill is seeking an ETL Developer for a Porto-based project with hybrid work (2x per week in the office). You will design, build, and operate ETL pipelines that import tariff and customs data via REST APIs, bulk downloads, and web crawling.

Collaborate with customs domain experts to ensure data quality and a robust relational model. The role requires 5+ years in ETL/data engineering, Python-based pipelines, strong SQL skills, and the ability to own data sources end-to-end.

Qualificações

• 5+ years of experience in ETL/Data engineering roles.
• Proven production ETL pipelines in Python (requests/httpx, BeautifulSoup, Scrapy, Playwright, pandas, SQLAlchemy).
• Experience consuming REST APIs and building web crawlers/scrapers with auth and throttling.
• Strong SQL and relational data modeling, ideally PostgreSQL.
• Ability to own data sources end-to-end from analysis to monitoring.
• Experience with XML, JSON, and CSV datasets at scale.

Responsabilidades

• Design, build, and operate ETL pipelines importing tariff and customs data via REST APIs, bulk downloads, and web scraping.
• Develop resilient web crawling logic for portals without APIs, including change detection, pagination, rate limiting, and recovery.
• Analyze source data structures per jurisdiction, including nomenclature hierarchies, duty measures, and legal references.
• Design and evolve the relational data model (PostgreSQL) balancing normalization and performance.
• Implement data quality checks, reconciliation, and data versioning for tariff changes over time.
• Monitor pipeline health and adapt pipelines to changing publication formats.
• Document data lineage, source peculiarities, and data mapping decisions.
• Collaborate with customs domain experts and engineering to align data model with requirements.

Conhecimentos

Python

SQL

Data modeling

Web crawling

Ferramentas

requests/httpx

BeautifulSoup

Scrapy

Playwright

pandas

SQLAlchemy

PostgreSQL

Descrição da oferta de emprego

We are growing. And we are looking for talented people.

At Decskill, we believe that technological excellence is driven by human talent.

We are an IT consulting company with more than 10 years of consolidated experience in the market, focused on building long-term relationships with both our clients and our people. Today, we are a community of over 800 professionals, working from Lisbon, Porto, and Madrid, contributing to impactful technology initiatives.

As part of the Astek Group, we combine a strong local culture with a global presence, being active in around 23 countries across 4 continents. This allows us to offer an international context, diverse challenges, and long-term career opportunities, while staying close to our teams.

We would like to meet an ETL Developer for a project with a hybrid working model in Porto, Portugal (2x/week at the office).

Responsabilities

• Design, build, and operate ETL pipelines that import tariff and customs nomenclature data from European and North American authorities via REST APIs, bulk file downloads, and web crawling/scraping
• Develop resilient web crawling logic for portals without APIs, including change detection, pagination handling, rate limiting, and graceful failure recovery
• Analyze source data structures per jurisdiction, including nomenclature hierarchies, duty measures, validity periods, legal references, and country-specific extensions
• Design and evolve the relational data model (PostgreSQL), balancing normalization, query performance, and divergent national schemas
• Implement data quality checks, reconciliation logic, and data versioning to accurately track tariff changes over time
• Monitor pipeline health, investigate data anomalies, and adapt pipelines when authorities change publication formats
• Document data lineage, source peculiarities, and data mapping decisions
• Collaborate with customs domain experts and the engineering team to align the data model with operational and regulatory requirements

Requirements

• 5+ years of experience as an ETL Developer, Data Engineer, BI Developer, or in a similar data engineering role.
• Proven experience building production ETL/data pipelines in Python (e.g., requests/httpx, BeautifulSoup, Scrapy, Playwright, pandas, SQLAlchemy)
• Hands‑on experience consuming REST APIs and building web crawlers/scrapers, including authentication, throttling, and handling unstable data sources
• Strong SQL skills and practical experience with relational data modeling, ideally using PostgreSQL
• Excellent analytical skills with the ability to understand complex, poorly documented data sources and identify their structure, semantics, and edge cases
• Strong interest in data modeling and database schema design
• Ability to own data sources end-to-end, from analysis and implementation to monitoring, working independently
• Experience working with XML, JSON, and CSV datasets at scale

Nice to Have

• Working knowledge of Java
• Experience with Apache NiFi or similar data integration/dataflow platforms
• Experience with data versioning and Slowly Changing Dimensions (SCD)

Are you looking for an environment that values curiosity and commitment? Here, your contribution has real impact and individual growth is taken seriously.

Find with us the right opportunity to grow!

What you can expect from us

• Long-term projects with national and international context
• Opportunities to grow technically and professionally
• A people-first culture, focused on transparency and trust
• Teams that value ownership, collaboration and stability

Your next challenge might start here.

At Decskill, we are committed to equal opportunities and non-discrimination. We promote a culture of diversity and inclusion, where recruitment and career progression are based solely on talent, regardless of age, gender, ethnicity, race, nationality, or any other form of discrimination incompatible with human dignity.