Senior Big Data Engineer — Data Lake & PySpark

act digital

Porto

Presencial

EUR 45 000 - 65 000

Tempo integral

14 dias+

Recebe mais respostas dos empregadores

Envia um currículo específico para a oferta em poucos minutos.

Resumo da oferta

act digital in Porto is looking for an experienced Data Engineer/BI Engineer to design and operate Data Lake pipelines using Python and PySpark, delivering scalable data solutions for analytics. You will collaborate with Data Engineers, BI Developers, Analysts, and stakeholders to ensure robust data delivery and performance optimization.

Strong SQL and data modeling foundations are essential, along with hands-on experience in Big Data, NoSQL, and DevOps practices.

Qualificações

5+ years of professional experience as a Data Engineer / BI Engineer.
Minimum 4 years of hands-on experience with Big Data and NoSQL technologies (Hadoop, HDFS, Hive, Kerberos, Starburst).
Experience developing Data Lake pipelines using Python and PySpark (Spark is a plus).
Solid knowledge of SQL and Data Modeling principles.
Experience with Visual Studio Code or similar IDEs.
Familiarity with DevOps concepts and CI/CD practices.
Experience with Git for source code management.
Experience with Jenkins for build automation.
Experience with XL Deploy and XL Release for deployment processes.
English level B2 or higher (mandatory).

Responsabilidades

Develop and maintain Data Lake pipelines using Python and PySpark.
Analyze business and technical requirements and propose scalable Data Lake architecture solutions.
Gather, document and manage application requirements within an Agile environment.
Design and implement data solutions ranging from simple to complex use cases, promoting automation and operational efficiency.
Maintain and enhance existing data processes while providing technical support to users.
Collaborate with Data Engineers, BI Developers, Data Analysts, Business Analysts, Software Developers and business stakeholders to deliver data-driven solutions.
Contribute throughout the full project lifecycle, including assessment, use case identification, data preparation, implementation and industrialization.
Identify opportunities to improve existing processes, with a strong focus on performance optimization.
Work within a cross-functional international team across Porto and Paris, contributing to knowledge sharing and continuous improvement of the data ecosystem.