Senior Data Engineer: Spark, Delta Lake & Databricks

InnoTech; Portugal; Presencial; EUR 45 000 - 75 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Mental health support

25 days vacation

Birthday/bridging day

Petsitting partnerships

Pet bereavement day

Team events

InnoSports initiatives

Sustainability initiatives

Resumo da oferta

InnoTech is seeking a data engineer to design and run scalable data solutions in Databricks. You will develop PySpark workflows, build ETL/ELT pipelines with Delta Lake, and ensure reliable batch and streaming processing.

The role emphasizes governance and platform reliability while collaborating with cross-functional teams. Ideal candidates bring strong SQL, Spark architecture knowledge, and EU work eligibility.

Qualificações

• Proven Databricks experience with Workspaces, Notebooks, Workflows, and Unity Catalog.
• Strong PySpark, Spark SQL and distributed workload optimisation skills.
• Advanced SQL with complex joins, windows and aggregations.
• Solid ETL/ELT, Delta Lake, CDC, and data modelling knowledge.
• CPLP nationality or EU work permit required.

Responsabilidades

• Develop and maintain scalable data solutions in the Databricks ecosystem.
• Build and optimise PySpark applications for large datasets.
• Design ETL/ELT pipelines using Delta Lake and Medallion Architecture.
• Orchestrate batch and streaming data workflows for reliability.
• Collaborate to improve data performance, governance, and platform reliability.

Conhecimentos

Databricks

PySpark

Spark SQL

SQL

Data modelling

ETL/ELT

CDC

Delta Lake

Unity Catalog

Spark architecture

Formação académica

BSc in Computer Science/Engineering

MSc in IT/Engineering

Ferramentas

Pandas

NumPy

Auto Loader

Descrição da oferta de emprego

InnoTech is seeking a data engineer to design and run scalable data solutions in Databricks. You will develop PySpark workflows, build ETL/ELT pipelines with Delta Lake, and ensure reliable batch and streaming processing.

The role emphasizes governance and platform reliability while collaborating with cross-functional teams. Ideal candidates bring strong SQL, Spark architecture knowledge, and EU work eligibility.