Data Engineer

Optiwisers; Portugal; Híbrido; EUR 45 000 - 65 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Competitive salary and benefits package

Career growth and continuous learning

Opportunity to work on complex data projects

Flexible work arrangements

Resumo da oferta

A leading data solutions provider in Portugal is seeking a Data Engineer with a focus on Databricks and Spark. You will be responsible for optimizing data pipelines, applying performance strategies, and ensuring reliable data processing for complex business requirements. The role offers flexibility in work arrangements with competitive salary and benefits, and opportunities for career growth and continuous learning.

Qualificações

• Proven experience as a Data Engineer working with large-scale data platforms.
• Strong hands-on experience with Databricks, Spark, and PySpark.
• Solid knowledge of Delta Lake and data lake best practices.

Responsabilidades

• Design, optimize, and operate scalable data platforms.
• Develop and maintain Airflow DAGs.
• Handle task retries, failure scenarios, and recovery strategies.

Conhecimentos

Databricks

Spark

PySpark

SQL

Azure Data Factory

Apache Airflow

Delta Lake

Data Modeling

Performance Optimization

Descrição da oferta de emprego

Over 30 years providing HR solutions to clients and candidates

Type: Full-time, long-term

We are looking for an experienced Data Engineer to join our growing data team and contribute to the design, optimization, and operation of scalable data platforms. You will work with modern data technologies, supporting complex business use cases and ensuring high-performance, reliable data pipelines.

Databricks & Spark

• Optimize slow-running Spark jobs to improve performance and cost efficiency.
• Implement storage optimization using Delta Lake features such as Z-ordering and partitioning.
• Manage and fine‑tune Databricks cluster configurations.
• Install and manage libraries across environments.
• Apply and maintain Medallion Architecture (Bronze, Silver, Gold layers).
• Translate complex business requirements into efficient data processing logic.

PySpark & SQL

• Apply partitioning and bucketing strategies for performance optimization.
• Handle skewed data and optimize large‑scale joins.
• Develop complex joins, advanced SQL queries, and window functions.
• Use broadcast joins and repartitioning where appropriate.
• Work with CTEs and structured/semi‑structured data, including JSON formats.
• Perform detailed requirement analysis for complex business processes.
• Develop, debug, and optimize ADF pipelines.
• Implement event‑based triggers and incremental data loading strategies.
• Create and manage EDF pipelines and orchestrate multiple dependent pipelines.
• Design reusable and generic ADF pipelines to support multiple use cases.

Airflow

• Develop and maintain Airflow DAGs.
• Write custom operators and sensors.
• Handle task retries, failure scenarios, and recovery strategies.
• Use XComs for inter‑task data sharing.
• Proven experience as a Data Engineer working with large‑scale data platforms.
• Strong hands‑on experience with Databricks, Spark, and PySpark.
• Solid knowledge of Delta Lake and data lake best practices.
• Experience with Azure Data Factory for pipeline orchestration.
• Strong experience with Apache Airflow.
• Advanced SQL skills and strong understanding of data modeling.
• Ability to analyze complex business requirements and translate them into technical solutions.
• Strong problem‑solving skills and attention to performance optimization.

Nice to Have

• Experience with cloud platforms (Azure preferred).
• Knowledge of CI/CD for data pipelines.
• Experience working in Agile environments.
• Familiarity with data governance and data quality frameworks.

What We Offer

• Opportunity to work on complex, high‑impact data projects.
• Modern data stack and exposure to best‑in‑class technologies.
• Competitive salary and benefits package.
• Career growth, continuous learning, and technical development.
• Hybrid or remote working model (depending on location).

Send your CV to [e-mail] Subject: Data Engineer (Banking)

Referrals increase your chances of interviewing at Optiwisers by 2x.

Seniority Level

Mid‑Senior level

Employment Type

Full-time

Job Function

Information Technology and Consulting

Industries

Banking