Senior Data Engineer - Databricks Lakehouse & AI Pipelines

AgileEngine; Warszawa; On-site; PLN 260,000 - 360,000; Full time

Benefits offered by this job

Growth without limits

Competitive compensation

Remote work 100% with flexible hours

Meaningful, modern projects

Collaborative culture

Well-being & support

Job summary

AgileEngine is seeking an experienced Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will design and operate batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture for analytics, reporting, and ML workloads.

You will migrate legacy ETL workloads to the Lakehouse, ensure data parity, and leverage AI tools like Claude and GitHub Copilot to speed up development.

Qualifications

• 4+ years of professional experience in data engineering with Spark and cloud data architectures.
• Hands-on experience with Databricks, Spark (PySpark), and Delta Lake.
• Strong SQL and Python skills, data modeling across Lakehouse patterns.

Responsibilities

• Design, build, and operate batch and streaming data pipelines on Databricks using PySpark, Delta Lake, and Databricks Workflows.
• Model and maintain a medallion architecture (bronze/silver/gold) for analytics, reporting, and ML consumption.
• Migrate legacy ETL workloads to Lakehouse with data parity and minimal disruption.

Skills

Apache Spark

Databricks

PySpark

SQL

Python

English (business)

Tools

Terraform

Azure DevOps

PostgreSQL

Dynatrace

CloudWatch

Job description

AgileEngine is seeking an experienced Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will design and operate batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture for analytics, reporting, and ML workloads.

You will migrate legacy ETL workloads to the Lakehouse, ensure data parity, and leverage AI tools like Claude and GitHub Copilot to speed up development.