Azure Databricks Data Engineer: Real-Time Pipelines & DevOps

Luxoft Poland; Polska; On-site; PLN 240,000 - 340,000; Full time

Benefits offered by this job

Private medical care

Dental care

Life insurance

MyBenefit program

Internal mobility

Job summary

Luxoft Poland is seeking an experienced Azure Databricks Developer to design and implement scalable data pipelines and cloud-based analytics solutions. You will work with data engineers, architects, and business teams on real-time streaming and ETL processes within the SaaS Data Layer for the new trading platform.

The role emphasizes Azure Databricks, Azure DevOps, Python, PySpark, and SQL, with a focus on security, governance, CI/CD automation, and continuous learning; opportunities for

Qualifications

• 10+ years experience in development.
• 3+ years of hands-on experience with Azure cloud services, including Databricks and DevOps.
• Strong proficiency Python and Pyspark or SQL for data transformation and analysis.
• Experience with real-time data streaming and event-driven architectures.
• Deep understanding of ETL/ELT workflows and big data processing frameworks
• Knowledge of Azure DevOps and CI/CD pipelines for automated deployments.
• Familiarity with data security, governance, and compliance best practices in cloud environments.
• Excellent problem-solving skills and ability to work in an agile development environment.
• Strong communication and collaboration skills.
• Agile
• Certifications such as Microsoft Certified: Azure Data Engineer Associate or Azure Solutions Architect.
• Experience with Power BI or other visualization tools.
• Knowledge of machine learning and AI-based analytics on Azure.

Responsibilities

• Design, implement, and optimize data pipelines in Azure Databricks.
• Develop ETL/ELT processes and real-time streaming solutions.
• Collaborate with cross-functional teams including data architects and BI developers to build integrated data solutions.
• Ensure data governance, security, and compliance across cloud data platforms.
• Automate deployment and monitoring of data workflows using CI/CD pipelines.
• Stay updated with emerging Azure technologies and propose performance and cost improvements.

Skills

Azure Databricks

Azure DevOps

Python

PySpark

SQL

Real-time streaming

ETL/ELT

CI/CD

Data governance & security

Agile

Communication

Machine learning basics

Tools

Azure Functions

Azure Service Bus

Power BI

Azure Log Analytics

CI/CD pipelines

Job description

Luxoft Poland is seeking an experienced Azure Databricks Developer to design and implement scalable data pipelines and cloud-based analytics solutions. You will work with data engineers, architects, and business teams on real-time streaming and ETL processes within the SaaS Data Layer for the new trading platform.

The role emphasizes Azure Databricks, Azure DevOps, Python, PySpark, and SQL, with a focus on security, governance, CI/CD automation, and continuous learning; opportunities for