Senior Azure Databricks Data Engineer - Real-Time Pipelines

Luxoft Poland; Poland; On-site; PLN 240,000 - 360,000; Full time

Benefits offered by this job

Private Medical & Dental care & Life보험

Job summary

Luxoft Poland is seeking an experienced Azure Databricks Developer to join our SaaS Data Layer team. You will design and implement scalable cloud data solutions, building data pipelines and real-time streaming across client environments on Azure.

You will collaborate with data engineers and business teams, implement governance and security, and drive CI/CD automation to ensure reliable deployments. Growth and training are provided by Luxoft.

Qualifications

• 10+ years experience in development.
• 3+ years hands-on experience with Azure Databricks and DevOps.
• Strong proficiency Python and PySpark or SQL for data transformation and analysis.
• Experience with real-time data streaming and event-driven architectures.
• Deep understanding of ETL/ELT workflows and big data processing frameworks.
• Knowledge of Azure DevOps and CI/CD pipelines for automated deployments.
• Familiarity with data security, governance, and compliance best practices in cloud environments.
• Excellent problem-solving skills and ability to work in an agile development environment.

Responsibilities

• Cloud expertise in Azure Databricks, Azure DevOps, Azure Log Analytics; knowledge of Azure network concepts, Azure Service Bus, Azure Functions is a plus.
• Develop, implement, and maintain data pipelines and ETL processes using Azure Databricks.
• Design and manage Azure Databricks for optimized data storage and retrieval.
• Collaborate with cross-functional teams to define and implement data integration solutions across multiple sources and platforms.
• Implement best practices for data governance, security, and compliance in cloud-based solutions.
• Monitor and troubleshoot performance issues in Azure-based applications and data pipelines.
• Automate deployment and monitoring processes using CI/CD pipelines.
• Stay updated with emerging Azure technologies and provide recommendations for improvements.

Skills

Azure Databricks

Azure DevOps

Python

PySpark

SQL

Real-time streaming

ETL/ELT

CI/CD pipelines

Data governance

Security & compliance

Agile collaboration

Job description

Luxoft Poland is seeking an experienced Azure Databricks Developer to join our SaaS Data Layer team. You will design and implement scalable cloud data solutions, building data pipelines and real-time streaming across client environments on Azure.

You will collaborate with data engineers and business teams, implement governance and security, and drive CI/CD automation to ensure reliable deployments. Growth and training are provided by Luxoft.