Azure Databricks Lead Engineer - Cloud Data Pipelines

Luxoft

Poland

Remote

PLN 180,000 - 300,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Luxoft is looking for an experienced Azure Databricks Developer to design and implement scalable cloud-based data solutions. You will build data pipelines, streaming, and analytics with Azure Databricks and DevOps, collaborating with data engineers and architects to optimize data processing and governance.

The role emphasizes CI/CD, security, and documentation, with opportunities for professional growth and training within Luxoft.

Qualifications

  • Strong cloud and data engineering capabilities with Azure Databricks and Azure DevOps.
  • Proficient in Python and PySpark or SQL for data transformation and analysis.
  • Experience with real-time data streaming and event-driven architectures.
  • Knowledge of ETL/ELT workflows and big data processing frameworks.
  • Familiarity with CI/CD pipelines and data security best practices.

Responsibilities

  • Develop, implement, and maintain data pipelines and ETL processes using Azure Databricks.
  • Design and manage Azure Databricks for optimized data storage and retrieval.
  • Collaborate with cross-functional teams to implement data integration across sources and platforms.
  • Implement data governance, security, and compliance in cloud solutions.
  • Monitor, troubleshoot, and optimize Azure-based data pipelines and applications.
  • Automate deployment and monitoring with CI/CD pipelines.
  • Stay updated with Azure technologies and propose improvements.
  • Produce architectural diagrams and technical documentation.

Skills

Azure Databricks
Azure DevOps
Python
PySpark
SQL
Real-time streaming
ETL/ELT
CI/CD pipelines
Data governance
Cloud security
Agile
Communication

Job description

Luxoft is looking for an experienced Azure Databricks Developer to design and implement scalable cloud-based data solutions. You will build data pipelines, streaming, and analytics with Azure Databricks and DevOps, collaborating with data engineers and architects to optimize data processing and governance.

The role emphasizes CI/CD, security, and documentation, with opportunities for professional growth and training within Luxoft.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Databricks Data Engineer Real-Time Pipelines DevOps
Azure Databricks Data Engineer Real-Time Pipelines DevOps

DXC Luxoft • Warszawa

On-site
PLN 150,000 - 230,000
Professional training
Senior Azure Databricks Engineer: Real-Time Data Pipelines
Senior Azure Databricks Engineer: Real-Time Data Pipelines

Luxoft • Województwo mazowieckie

On-site
PLN 180,000 - 300,000
Professional training
Career growth opportunities
Luxoft employee benefits
Azure Databricks Developer
Azure Databricks Developer

DXC Luxoft • Warszawa

On-site
PLN 150,000 - 230,000
Professional training
Azure Databricks Developer
Azure Databricks Developer

Luxoft • Województwo mazowieckie

On-site
PLN 180,000 - 300,000
Professional training
Career growth opportunities
Luxoft employee benefits
Senior Azure Databricks Developer
Senior Azure Databricks Developer

Luxoft • Poland

Remote
PLN 180,000 - 300,000
Senior Data Engineer: Azure, Databricks & Data Pipelines
Senior Data Engineer: Azure, Databricks & Data Pipelines

Luxoft • Wrocław

On-site
PLN 180,000 - 280,000
Senior BigData Platform Engineer - Azure, Spark, Databricks
Senior BigData Platform Engineer - Azure, Spark, Databricks

Luxoft • Wrocław

On-site
PLN 140,000 - 210,000
Private Medical & Dental care
Life Insurance covered
Internal Mobility program
Senior Big Data Platform Engineer (Azure/Databricks)
Senior Big Data Platform Engineer (Azure/Databricks)

Luxoft • Wrocław

On-site
PLN 180,000 - 360,000
Senior BigData Platform Engineer - Azure, Spark, Databricks
Senior BigData Platform Engineer - Azure, Spark, Databricks

Luxoft • Kraków

On-site
PLN 320,000 - 480,000
Private Medical
Dental care
Life Insurance
+1
Cloud Platform Engineer - Databricks/Big Data
Cloud Platform Engineer - Databricks/Big Data

Luxoft • Kraków

On-site
PLN 120,000 - 180,000
Top locations