Data Engineer

Hexa Consulting; Porto; Presencial; EUR 22 000 - 28 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Private health insurance

Mental health support

Training opportunities

Flexible work

Childcare vouchers

Team events

Upskilling in AI governance

Resumo da oferta

Hexa Consulting is seeking a PySpark Developer/Data Engineer to design, develop, and optimize large-scale data processing solutions for a financial services client in Porto. You will build scalable batch and streaming pipelines using PySpark and Spark, optimize workloads, and ensure data quality across diverse sources.

The role emphasizes translating requirements into efficient designs, adopting best practices, and collaborating with cross-functional teams in an Agile setup.

Qualificações

• Proven PySpark/Data Engineer background with Spark expertise.
• Strong problem solving, autonomy, and data quality focus.

Responsabilidades

• Design, develop and maintain scalable batch and streaming data pipelines.
• Optimize Spark workloads: partitioning, shuffle reduction, memory tuning and data skew mitigation.
• Integrate and validate large heterogeneous datasets from databases, APIs, files and streams.
• Translate requirements into data flows and technical designs.
• Apply software engineering best practices: clean code, reviews, tests, maintainable delivery.

Conhecimentos

Python

PySpark

Spark SQL

DataFrames & Datasets

Spark architecture

SQL (complex queries)

Parquet

SQL/NoSQL databases

APIs

Kafka streams

Git

PyTest

Unit/Integration testing

Databricks or EMR

Ferramentas

Databricks

EMR

Descrição da oferta de emprego

We are looking for a PySpark Developer/Data Engineer to support a financial services sector organization in the development, optimization and industrialization of large-scale data processing solutions for business-critical application environments.

WHAT YOU WILL DO

• Design, develop and maintain scalable batch and streaming data pipelines using PySpark and Apache Spark.
• Optimize existing Spark workloads by improving partitioning, reducing shuffle operations, tuning memory utilization and resolving data skew issues.
• Integrate, transform and validate large heterogeneous datasets from databases, APIs, file-based sources and streaming platforms.
• Translate functional requirements into efficient technical designs, data flows and implementation approaches.
• Apply software engineering best practices, including clean code principles, code review, automated testing and maintainable delivery standards.
• Deploy, automate and monitor data pipelines using orchestration and delivery tools.
• Collaborate with data engineers, data scientists, architects and cross-functional teams in an Agile environment.

WHAT WE ARE LOOKING FOR

• Professional experience in software development or data engineering, with proven specialization in PySpark and Apache Spark.
• Strong analytical capability, technical rigor and autonomy in solving data processing and performance challenges.
• Ability to communicate clearly with technical and functional stakeholders and contribute to collaborative delivery.
• Commitment to software quality, maintainability, optimization and reliable data engineering practices.

TECHNICAL SKILLS

• Python
• PySpark
• Spark SQL
• DataFrames and Datasets API
• Spark architecture, including Driver, Workers, DAG, lazy evaluation and caching mechanisms
• SQL and complex query optimization
• Parquet
• SQL and NoSQL databases
• APIs
• Kafka streams
• Git
• PyTest
• Unit testing and integration testing
• Databricks or EMR

NICE TO HAVE

• Experience with Prefect or additional workflow orchestration tools.
• Experience with advanced monitoring and performance analysis of distributed data processing workloads.

COMPENSATION TRANSPARENCY

• Base salary range: EUR 22,000 - EUR 28,000 gross/year
• Total compensation: up to EUR 34,000/year
• The salary range is based on objective and gender-neutral criteria, including required competencies, professional experience, level of responsibility and role requirements. Final compensation will be determined according to the candidate's profile and position requirements.

WHAT WE OFFER

• Private health insurance from the first day, with optional family extension.
• Confidential mental health support through free access to psychological support, content and live sessions.
• Continuous training and certification opportunities to support long-term professional development.
• Flexibility whenever the role and context allow it, supporting sustainable delivery and daily balance.
• Childcare vouchers and practical support measures designed to help balance personal and professional life.
• Team moments, shared milestones and leadership presence when it matters most.
• Upskilling opportunities in quality management, information security, AI governance and cybersecurity best practices.

EQUAL OPPORTUNITY

Hexa is committed to equal opportunity and inclusive recruitment practices. We value diverse backgrounds, perspectives and experiences, and our approach is aligned with the principles of the Portuguese Diversity Charter.

HEXA LIFE

At Hexa, every professional is a Builder. We work with honesty, mutual support and agility, creating strong professional relationships and contributing to projects, solutions and teams with responsibility, transparency and continuous improvement.