Data Engineer / Senior Data Engineer with Azure, Databricks and Microsoft Fabric

DataArt; Poland; Hybrid; PLN 200,000 - 320,000; Full time

Benefits offered by this job

Vacation days 26+

Health and life insurance (Luxmed)

MyBenefit with Multisport

Psychological support service

English language classes from day one

Access to learning platforms (O'Reilly

Flexible workplace: office/home/hybrid

Tech Skills Mentoring Program

Public speaker/mentoring opportunities

Fully paid idle time (bench)

Certification reimbursement (AWS, GCP,

Microsoft)

Job summary

DataArt is hiring a Senior Data Engineer to design and build a modern cloud data platform on Microsoft Azure. You will own ingestion, transformation, and curation layers while collaborating with business and technical stakeholders.

You will work with Azure Databricks, Microsoft Fabric, and Delta Lake, delivering scalable data solutions, governance, and analytics products in a regulated financial services context.

Qualifications

• 6+ years of experience in Data Engineering.
• Hands-on with Azure Data Factory, Azure Databricks, and Azure Data Lake Storage Gen2.
• Strong programming in Python, SQL, and PySpark.
• Experience implementing Delta Lake and medallion architectures.
• Experience with Git and Azure DevOps, including CI/CD pipelines for data solutions.
• Experience working with SQL Server environments.
• Proven ability to communicate with technical and business stakeholders in English.
• Experience taking ownership of architecture or platform design decisions.

Responsibilities

• Design and develop data ingestion pipelines into a governed cloud data platform.
• Build scalable data engineering solutions using Azure Databricks and Microsoft Fabric.
• Implement medallion architecture patterns and reusable engineering frameworks.
• Develop curated data products and analytical datasets for reporting.
• Apply AI-assisted development practices while ensuring quality and governance.
• Contribute to data modelling, metadata management, and analytics solutions.
• Support migration of reporting workloads from legacy platforms.
• Collaborate with stakeholders to translate business requirements into technical designs.
• Provide design guidance and participate in code reviews.

Skills

Azure Data Factory

Azure Databricks

Azure Data Lake Gen2

Python

SQL

PySpark

Delta Lake

Medallion Architecture

Git

Azure DevOps

CI/CD

SQL Server

AI coding assistants

English communication

Tools

Microsoft Fabric

Power BI

Purview

Databricks

Job description

Technology stack

Azure Data Factory, Azure Databricks, Microsoft Fabric, Azure Data Lake Storage Gen2, Delta Lake, Apache Spark, PySpark, Python, SQL Server, Azure DevOps, Git, Power BI, Microsoft Purview, GitHub Copilot

Client

Our client is a UK financial services group operating in a highly regulated environment.

Project overview

This project focuses on building a modern enterprise data and analytics platform on Microsoft Azure as part of a large scale modernization program. A legacy policy administration system is being replaced with a SaaS solution, while reporting and analytics capabilities are being migrated from a traditional SQL Server environment to Databricks and Microsoft Fabric. The platform will support governed data products, advanced analytics, and AI assisted engineering practices.

Position overview

We are looking for a Senior Data Engineer with strong experience in Azure, Databricks, and Microsoft Fabric to help design and build a modern cloud data platform. You will contribute to the ingestion, transformation, and curation layers of the platform while collaborating with technical and business stakeholders. In this role, you will also support engineering standards, platform design decisions, and the delivery of scalable data solutions.

Responsibilities

• Design and develop data ingestion pipelines from business systems into a governed cloud data platform.
• Build scalable data engineering solutions using Azure Databricks and Microsoft Fabric.
• Implement medallion architecture patterns and reusable engineering frameworks.
• Develop curated data products and analytical datasets for business reporting and decision making.
• Apply AI assisted development practices while ensuring quality, governance, and auditability.
• Contribute to data modelling, semantic models, metadata management, and analytical solutions.
• Support the migration of reporting and analytics workloads from legacy platforms.
• Collaborate with stakeholders to translate business requirements into technical solutions.
• Provide design guidance, establish engineering standards, and participate in code reviews.

Requirements

• 6+ years of experience in Data Engineering.
• Hands on experience with Azure Data Factory, Azure Databricks, and Azure Data Lake Storage Gen2.
• Strong programming experience with Python, SQL, and PySpark.
• Experience implementing Delta Lake and production lakehouse or medallion architectures.
• Experience with Git and Azure DevOps, including CI/CD pipelines for data solutions.
• Experience working with SQL Server environments.
• Practical experience using AI coding assistants within software delivery projects.
• Ability to communicate effectively in English with technical and business stakeholders.
• Experience taking ownership of architecture or platform design decisions.
• Experience reviewing code and establishing engineering standards.
• Experience gathering requirements and collaborating directly with business stakeholders.

Nice to have

• Experience implementing Microsoft Fabric in production environments.
• Microsoft certifications including DP 203, DP 600, DP 700, AZ 900, or DP 900.
• Databricks Data Engineer Associate or Databricks Data Engineer Professional certification.
• Experience with dimensional modelling, including star schema, snowflake schema, or Data Vault.
• Experience with semantic modelling in Power BI or Microsoft Fabric.
• Knowledge of Microsoft Purview, data lineage, and metadata management.
• Experience working in regulated financial services environments.

What We Offer:

• Vacation days: Up to 26 business days per year.
• 10 illness/special days off per year (fully paid, no medical papers needed) for all contract types.
• Health and life insurance (Luxmed).
• MyBenefit platform with Multisport option.
• Internal psychological support service.
• English language classes from the first working day.
• Access to external learning platforms: O'Reilly, LinkedIn Learning, Udemy, and a wide catalog of diverse internal training.
• Flexible workplace: work from the office, from home, or choose a hybrid option.
• Tech Skills Mentoring Program.
• Opportunities to develop as a public speaker, mentor, or technical interviewer.
• Fully paid idle (bench) when not involved in a project.
• Certification reimbursement (AWS, GCP, Microsoft, etc.).