Senior Azure Databricks Data Engineer, Cloud Lakehouse Lead

Aura Recruitment Solutions; Poland; Remote; PLN 101,000 - 195,000; Full time

Job summary

Aura Recruitment Solutions is seeking a Senior Data Engineer – Azure Databricks to design, develop, and optimize enterprise data platforms. You will work with Azure Databricks, PySpark, Python, and SQL to build scalable pipelines and modern cloud-based data ecosystems.

The role emphasizes CI/CD with Azure DevOps, data lake architectures, and collaboration with data architects and stakeholders. Remote work is offered with strong Azure Cloud focus.

Qualifications

• 8+ years of experience in Data Engineering in enterprise environments.
• Strong hands-on with Azure Databricks, PySpark, Python, and SQL.
• Experience with Azure Data Factory (ADF), ADLS Gen2, and core Microsoft Azure cloud data services.
• CI/CD experience using Azure DevOps across multi-stage environments (Dev, QA, Prod).
• Advanced SQL with performance tuning.

Responsibilities

• Data Pipeline Engineering: design, build, and maintain scalable ETL/ELT pipelines using Azure Databricks, PySpark, Python, and SQL.
• Orchestration & Integration: build data ingestion workflows with Azure Data Factory to integrate data into ADLS Gen2.
• Lakehouse Architecture & Modeling: design scalable lakehouse and medallion architectures to support analytics and reporting.
• Performance & Cost Optimization: audit and optimize Databricks jobs, Spark workloads, and queries for performance and cost.
• CI/CD & DevOps: develop automated CI/CD deployment pipelines using Azure DevOps across environments.
• Platform Governance & Quality: ensure data quality, security, governance, and troubleshoot production data issues.
• Engineering Standards & Leadership: lead design reviews and mentor junior/mid-level engineers.

Skills

Azure Databricks

PySpark

Python

SQL

Azure Data Factory

Azure Cloud

CI/CD

Tools

Azure DevOps

Terraform

Job description

Aura Recruitment Solutions is seeking a Senior Data Engineer – Azure Databricks to design, develop, and optimize enterprise data platforms. You will work with Azure Databricks, PySpark, Python, and SQL to build scalable pipelines and modern cloud-based data ecosystems.

The role emphasizes CI/CD with Azure DevOps, data lake architectures, and collaboration with data architects and stakeholders. Remote work is offered with strong Azure Cloud focus.