Remote Senior Data Engineer - Databricks Lakehouse

AgileEngine; Lublin; On-site; PLN 180,000 - 320,000; Full time

Benefits offered by this job

Growth without limits

Competitive compensation

Flexibility: 100% remote with flexible

Meaningful modern projects

Collaborative culture

Well-being & support

Job summary

AgileEngine is seeking a Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture for analytics, reporting, and ML.

You will migrate legacy workloads, ensure data parity, and collaborate with DevOps, platform, and analytics teams while leveraging AI tools like Claude and GitHub Copilot to accelerate development.

Qualifications

• 4+ years of data engineering experience with Spark and cloud data architectures.
• Hands-on with Databricks, Spark (PySpark), and Delta Lake.
• Advanced SQL and Python with strong data modeling skills.
• Experience with streaming using Structured Streaming, Kafka, or Event Hubs.
• Experience with workflow orchestration (Databricks Workflows, Airflow, or Azure Data Factory).
• Experience with legacy platform migrations, ETL modernization, or maintaining data hygiene.

Responsibilities

• Design, build, and operate batch and streaming data pipelines on Databricks using PySpark, Delta Lake, and Databricks Workflows.
• Model and maintain a medallion architecture (bronze/silver/gold) for analytics and ML.
• Migrate legacy ETL workloads to Lakehouse with minimal disruption.
• Use AI tools to accelerate development and prototyping.
• Write clean Python and SQL, with testing and documentation.
• Tune Spark jobs and Delta tables for performance and cost.
• Implement data governance, quality, and lineage controls.
• Collaborate with DevOps and analytics engineers on observability and security.

Skills

Apache Spark

Databricks

PySpark

Delta Lake

SQL

Python

Data Modeling

ETL Modernization

Cloud Platforms

Workflow Orchestration

Education

Bachelor's degree in Computer Science or Engineering

Tools

Databricks Workflows

Airflow

Azure Data Factory

dbt

Terraform

Job description

AgileEngine is seeking a Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture for analytics, reporting, and ML.

You will migrate legacy workloads, ensure data parity, and collaborate with DevOps, platform, and analytics teams while leveraging AI tools like Claude and GitHub Copilot to accelerate development.