Data Engineer / Big Data

Decskill; Porto; Híbrido; EUR 45 000 - 75 000; Tempo integral

Resumo da oferta

Decskill is seeking a Data Engineer to join a project with a hybrid working model in Porto, Portugal, collaborating with international teams. Design, develop, and maintain Data Lake and Big Data pipelines using Python and PySpark, building scalable data architectures and robust data governance.

You will work with Data Engineers, BI Developers, Data Analysts, and Software Developers across multinational contexts, contributing to data-driven solutions in Agile environments.

Qualificações

• 5+ years of experience in Data Engineering.
• 4+ years of hands-on experience with Big Data and NoSQL technologies.
• Strong experience with Hadoop ecosystem technologies including Hadoop, HDFS, Hive, Kerberos, and Starburst.
• Solid experience with Python, PySpark, and Spark.
• Strong knowledge of SQL and Data Modeling principles.
• Knowledge of DevOps practices and CI/CD concepts.
• Experience with Git for source code management.
• Experience with Jenkins for build automation.
• Experience with XL Deploy and XL Release.
• Experience working with Visual Studio Code or similar environments.
• Experience working in Agile environments using Scrum and/or Kanban.
• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management abilities.
• Ability to work autonomously, proactively, and within international distributed teams.

Responsabilidades

• Design, develop, and maintain Data Lake and Big Data pipelines using Python, PySpark, and related technologies.
• Analyze business and technical requirements and propose scalable Data Architecture Solutions.
• Build, optimize, and industrialize data processing workflows to improve efficiency and automation.
• Collaborate with Data Engineers, BI Developers, Data Analysts, Software Developers, and business stakeholders across international teams.
• Develop and support data-driven solutions in an Agile environment.
• Monitor, troubleshoot, and maintain existing data platforms and processes.
• Contribute to data initiatives from assessment and use case identification through implementation and production deployment.
• Identify opportunities for performance optimization and continuous improvement.
• Ensure best practices in source code management, deployment processes, and data governance.

Conhecimentos

Data Engineering

Big Data

Python

PySpark

SQL

DevOps

Git

Jenkins

XL Deploy

XL Release

Agile

Communication

Ferramentas

Hadoop

HDFS

Hive

Kerberos

Starburst

Python

PySpark

Spark

Git

Jenkins

VS Code

Descrição da oferta de emprego

We are growing. And we are looking for talented people.

At Decskill, we believe that technological excellence is driven by human talent.

We are an IT consulting company with more than 10 years of consolidated experience in the market, focused on building long-term relationships with both our clients and our people. Today, we are a community of over 800 professionals, working from Lisbon, Porto, and Madrid, contributing to impactful technology initiatives.

As part of the Astek Group, we combine a strong local culture with a global presence, being active in around 23 countries across 4 continents. This allows us to offer an international context, diverse challenges, and long-term career opportunities, while staying close to our teams.

We would like to meet a Data Engineer / Big Data for a project with a hybrid working model in Porto, Portugal (2x/week at the office).

Responsibilities:

• Design, develop, and maintain Data Lake and Big Data pipelines using Python, PySpark, and related technologies.
• Analyze business and technical requirements and propose scalable Data Architecture Solutions.
• Build, optimize, and industrialize data processing workflows to improve efficiency and automation.
• Collaborate with Data Engineers, BI Developers, Data Analysts, Software Developers, and business stakeholders across international teams.
• Develop and support data-driven solutions in an Agile environment.
• Monitor, troubleshoot, and maintain existing data platforms and processes.
• Contribute to data initiatives from assessment and use case identification through implementation and production deployment.
• Identify opportunities for performance optimization and continuous improvement.
• Ensure best practices in source code management, deployment processes, and data governance.

Requirements:

• 5+ years of experience in Data Engineering.
• 4+ years of hands-on experience with Big Data and NoSQL technologies.
• Strong experience with Hadoop ecosystem technologies, including Hadoop, HDFS, Hive, Kerberos, and Starburst.
• Solid experience with Python, PySpark, and Spark.
• Strong knowledge of SQL and Data Modeling principles.
• Knowledge of DevOps practices and CI/CD concepts.
• Experience with Git for source code management.
• Experience with Jenkins for build automation.
• Experience with XL Deploy and XL Release.
• Experience working with Visual Studio Code or similar development environments.
• Experience working in Agile environments using Scrum and/or Kanban.
• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management abilities.
• Ability to work autonomously, proactively, and within international distributed teams.

Nice to Have

• Experience in performance tuning, query optimization, shell scripting, and execution plan analysis.
• Knowledge of Power BI, Power Apps, and Power Automate.
• Experience with REST APIs.
• Knowledge of Azure, SSIS, and SSAS.
• Experience administering SharePoint Online and Dataverse environments.

Are you looking for an environment that values curiosity and commitment? Here, your contribution has real impact and individual growth is taken seriously.

Find with us the right opportunity to grow!

What you can expect from us

• Long-term projects with national and international context
• Opportunities to grow technically and professionally
• A people-first culture, focused on transparency and trust
• Teams that value ownership, collaboration and stability

Send us your CV in English to [e-mail] with reference “MA/ DEBG”.

Your next challenge might start here.

At Decskill, we are committed to equal opportunities and non-discrimination. We promote a culture of diversity and inclusion, where recruitment and career progression are based solely on talent, regardless of age, gender, ethnicity, race, nationality, or any other form of discrimination incompatible with human dignity.