Lead Data Engineer - Azure Databricks & GenAI

your Jared; Wrocław; Hybrid; PLN 180,000 - 260,000; Full time

Benefits offered by this job

Health insurance

Learning platforms (LinkedIn Learning,

English classes

Relocation support (abroad)

Job summary

Your Jared in Wrocław, Poland is seeking a Senior/Lead Data Software Engineer to migrate large-scale data workflows to an Azure-native Databricks environment and build robust PySpark/Delta ETL pipelines. You will optimize data quality, automate deployment, and support GenAI integration and governance initiatives.

The role requires 3+ years in data engineering, strong Python and Spark skills, and experience with Delta tables, Docker, Kubernetes, Terraform, and GitLab CI/CD.

Qualifications

• 3+ years in data engineering roles.
• Proficiency in Python and Spark.
• Azure Databricks and PySpark experience.
• Delta tables knowledge for ML-ready datasets.
• Familiarity with data governance frameworks.

Responsibilities

• Migrate and optimize 500+ data jobs to Azure Databricks.
• Build and maintain PySpark/Delta ETL/ELT pipelines.
• Tune ML models in Azure environments using Spark/Delta.
• Automate deployment and operations with Terraform and GitLab CI/CD.
• Support data governance and regulatory compliance initiatives.

Skills

Python

Spark

Azure Databricks

PySpark

Data governance

Tools

Docker

Kubernetes

AKS

Terraform

GitLab CI/CD

Job description

Your Jared in Wrocław, Poland is seeking a Senior/Lead Data Software Engineer to migrate large-scale data workflows to an Azure-native Databricks environment and build robust PySpark/Delta ETL pipelines. You will optimize data quality, automate deployment, and support GenAI integration and governance initiatives.

The role requires 3+ years in data engineering, strong Python and Spark skills, and experience with Delta tables, Docker, Kubernetes, Terraform, and GitLab CI/CD.