Senior Azure Data Engineer (PySpark/Python)

act digital; Porto; Hybrid; EUR 45,000 - 75,000; Full time

Job summary

act digital is seeking an experienced Data Engineer to design, develop and maintain scalable data solutions in Microsoft Azure. You will build reliable data pipelines using Python, PySpark and Spark, delivering business value into production while meeting security and reliability standards.

The role demands 5+ years in data engineering, strong knowledge of data modelling and lifecycle concepts, and excellent English.

Qualifications

• 5+ years of commercial experience in Data Engineering at senior level.
• Hands-on experience with Microsoft Azure and Azure-based data solutions.
• Strong proficiency in Python.
• Experience with Spark / PySpark or other distributed data processing frameworks.
• Understanding of data engineering best practices, patterns, and data lifecycle.
• Knowledge of data concepts, structures, and data modelling.
• Experience designing data to support different data consumption and business needs.
• Experience applying Test-Driven Development (TDD).
• Good English, written and spoken.
• Availability to work 1 day per week from the office in Matosinhos.

Responsibilities

• Design, develop, and maintain scalable data solutions using Azure and modern Data Engineering technologies.
• Deliver business value into Production, ensuring data solutions meet business and technical requirements.
• Develop and optimize data processing pipelines using Python and PySpark / Apache Spark.
• Apply Data Engineering best practices to ensure code quality, scalability, maintainability, and performance.
• Design appropriate data structures and data models according to business and technical requirements.
• Build and provision data solutions that support different consumption needs and use cases.
• Ensure data solutions are production-ready, reliable, and scalable.
• Apply TDD and automated testing practices throughout the development lifecycle.
• Collaborate with technical and business stakeholders to translate requirements into effective data solutions.

Skills

Data Engineering

Python

Data lifecycle

TDD

Data concepts and modelling

English

Tools

Microsoft Azure

Apache Spark

PySpark

Job description

act digital is seeking an experienced Data Engineer to design, develop and maintain scalable data solutions in Microsoft Azure. You will build reliable data pipelines using Python, PySpark and Spark, delivering business value into production while meeting security and reliability standards.

The role demands 5+ years in data engineering, strong knowledge of data modelling and lifecycle concepts, and excellent English.