Senior Data Engineer: Databricks, Spark & Azure Lakehouse

Enfint

United States

Remote

USD 150,000 - 210,000

Full time

6 days ago

Be an early applicant

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Annual holiday

Private healthcare insurance

Dental support

Travel insurance

Life insurance

Multisport card

Training Center courses

Job summary

Luxoft is seeking an experienced Data Engineer to develop and operate Databricks and Spark pipelines on Azure, focusing on reliability, freshness, and dataset quality against SLAs.

You'll design Delta Lake lakehouse layers, implement ETL/ELT with ADF or Airflow, tune Spark performance, and ensure lineage with Unity Catalog.

This role requires 7+ years in data engineering, expertise in Python/SQL, and Azure data services; relocation may be available.

Qualifications

7+ years of experience in data engineering.
Experience with Databricks / Apache Spark and Azure data services.
Strong Python and SQL programming skills.
Knowledge of Delta Lake and data governance.

Responsibilities

Develop and operate Databricks / Spark pipelines using PySpark and SQL.
Design Delta Lake lakehouse layers with bronze-silver-gold architecture.
Build ETL/ELT jobs and orchestration with ADF and Airflow.
Implement data-quality checks and validation against SLAs.
Tune Spark performance and manage cost with autoscaling and Photon.
Document data lineage and changes using Unity Catalog.

Skills

Databricks / Apache Spark

Python

SQL

Azure data services

Delta Lake

Airflow

Unity Catalog

Git / CI/CD

Education

Bachelor’s degree in Computer Science/Engineering/Information Systems

Tools

Azure Data Factory

Delta Live Tables

Azure Synapse/SQL DB

Airflow

Unity Catalog

Job description

Luxoft is seeking an experienced Data Engineer to develop and operate Databricks and Spark pipelines on Azure, focusing on reliability, freshness, and dataset quality against SLAs.

You'll design Delta Lake lakehouse layers, implement ETL/ELT with ADF or Airflow, tune Spark performance, and ensure lineage with Unity Catalog.

This role requires 7+ years in data engineering, expertise in Python/SQL, and Azure data services; relocation may be available.

Get your free, confidential resume review.

or drag and drop your file here.