Senior Data Engineer (BigData)

act digital; Lisboa; Presencial; EUR 40 000 - 60 000; Tempo integral

Resumo da oferta

act digital is looking for a Data Engineer to join their Data & Referentials team in Lisbon, with a strong focus on Big Data projects. The role involves implementing solutions using PySpark, maintaining code quality, and acting as a key contributor to technical projects.

Candidates should possess proven expertise in Hadoop, Spark, Scala, and PySpark, along with strong Java development skills. The ideal candidate will thrive in a collaborative environment, ensuring both quality and innovative solutions are at the forefront of development.

Qualificações

• Proven expertise in Hadoop, Spark, Scala, and PySpark.
• Significant professional experience in the Big Data domain.
• Excellent command of Java, including advanced development skills.

Responsabilidades

• Implement solutions using PySpark.
• Design, build, and maintain efficient, reusable, and reliable code.
• Participate in the definition and assessment of technical architectures.

Conhecimentos

Hadoop

Spark

Scala

PySpark

Java development

Ferramentas

Git

Jenkins

Azure

AWS

GCP

Descrição da oferta de emprego

We are looking for a Data Engineer to join a Data & Referentials team, with a strong focus on Big Data projects, scalable solution development, and continuous improvement of existing systems.

This role involves coordinating and supporting technical projects, ensuring the application of best development practices, exploring innovative solutions, and assessing potential enhancements to ongoing systems. The professional will also contribute to status meetings, identify new project opportunities, suggest improvements to current processes and initiatives, define the technical stack for new solutions, and evaluate the existing technical stack of active projects.

Responsibilities

• Implement solutions using PySpark
• Design, build, and maintain efficient, reusable, and reliable code
• Ensure the best possible performance, quality, and responsiveness of solutions
• Identify bottlenecks and bugs, and devise solutions to mitigate and address them
• Help maintain code quality, structure, and best development practices
• Participate in the definition and assessment of technical architectures and stacks
• Support continuous improvement initiatives across data and Big Data projects

Requirements

• Proven expertise in Hadoop, Spark, Scala, and PySpark
• Significant professional experience in the Big Data domain
• In-depth knowledge of the Hadoop, Spark, Scala, and PySpark ecosystem
• Excellent command of Java, including advanced development and maintenance skills
• Strong development skills in Scala or PySpark
• Experience rewriting Java components in Scala or PySpark
• Good knowledge of data engineering, systems engineering, software engineering, and Big Data architecture principles
• Ability to work independently and as part of a team
• Excellent communication and problem-solving skills

Technical skills considered a plus

• Data manipulation, data modeling, and SQL
• Java, Spring, Scala, Spark, PySpark, Python, and Maven
• Hadoop, HDFS, Hive, HBase, CDP, Ambari, Ranger, Kerberos
• Git, Jenkins, XL Deploy, and XL Release
• Experience with cloud provider ecosystems: Azure, AWS, or GCP
• First experience with GCP / BigQuery will be valued

Ideal profile

• Strong focus on quality and development best practices
• Analytical mindset and critical thinking
• Interest in Big Data environments and scalable solutions
• Proactive approach to identifying technical and functional improvements