Senior Data Engineer & Application Developer — Azure Spark

Luxoft; United Kingdom; Remote; GBP 70,000 - 120,000; Full time

Job summary

Luxoft is seeking a senior data engineer to design and deliver large-scale ETL/ELT pipelines on Azure Databricks, enabling ML and BI workloads. You will work across data sources and formats to build reliable data products and dashboards.

You will mentor junior engineers, shape data architecture, and implement best practices for data integration and quality. The role demands strong SQL, Python, Spark, and API skills, and a collaborative, cross-functional approach.

Qualifications

• 7+ years of experience in data engineering and ETL/ELT to deliver quality analytics solutions.
• 5+ years Python development and API design experience (GraphQL, REST).
• 2+ years Redis-backed state management and event-driven processing.
• 2+ years Apache Spark (PySpark/Spark SQL) and cloud data platforms (Azure Databricks).
• 2+ years SQL with complex schemas and query optimization.
• Experience building LLM integrations and DevOps automation with Terraform.

Responsibilities

• Design and develop ETL/ELT solutions on Azure Databricks and Spark-based platforms for analytics and ML.
• Consume data from RDBMS, APIs, FTPs and various formats (Excel, CSV, JSON, Parquet) and ensure quality.
• Write advanced SQL and optimize queries for performance.
• Mentor data engineers and participate in architectural evolution.

Skills

Python

PySpark/Spark SQL

Azure Databricks

SQL

ETL/ELT design

APIs (GraphQL/REST)

Redis state management

Terraform

Education

Bachelor's degree in Computer Science or related field

Tools

Snowflake

GitHub Actions

Service Bus

Job description

Luxoft is seeking a senior data engineer to design and deliver large-scale ETL/ELT pipelines on Azure Databricks, enabling ML and BI workloads. You will work across data sources and formats to build reliable data products and dashboards.

You will mentor junior engineers, shape data architecture, and implement best practices for data integration and quality. The role demands strong SQL, Python, Spark, and API skills, and a collaborative, cross-functional approach.