Hybrid Azure Databricks Lead Architect

EPAM Systems; Greater London; Hybrid; GBP 110,000 - 150,000; Full time

Benefits offered by this job

EPAM Employee Stock Purchase Plan (ES?

Life assurance

Income protection

Critical illness cover

Private medical insurance

Dental care

Employee Assistance Program

Pension plan

Cyclescheme

Techscheme

Season ticket loans

Free Wednesday lunch in-office

On-site massages

Regular social events

Learning and development opportunities

In-house training and coaching

Professional certifications

Discretionary annual bonus

Discretionary Long-Term Incentive (LTI

Job summary

EPAM Systems in London, hybrid role, seeks a Lead Databricks Engineer to design and lead enterprise-scale Azure and Databricks data platforms for advanced analytics and real-time processing. You will own platform architecture, governance, cost optimization, and ensure delivery of high-quality solutions in a complex data-driven environment.

You will lead architecture, enforce standards, optimize pipelines, mentor developers, and collaborate with senior stakeholders to align platform initiatives

Qualifications

• 8+ years of data engineering experience.
• 3+ years in a tech lead role.
• Enterprise-scale Databricks and Azure data platform expertise.
• Strong PySpark, Spark Structured Streaming and Delta Lake skills.
• Proficient Python and SQL, with performance tuning experience.
• Experience with data governance and observability frameworks.

Responsibilities

• Lead architecture design for Azure Databricks workloads.
• Define platform standards with governance and security.
• Build and optimize streaming pipelines with Delta Live Tables.
• Collaborate with cross-functional teams to translate requirements.
• Drive performance, cost optimization and monitoring.
• Provide engineering leadership across delivery and production support.
• Mentor developers and establish best practices.
• Participate in technical decisions for scalability and maintainability.
• Ensure observability and QA for critical pipelines.
• Engage stakeholders on platform strategy.

Skills

Data engineering

Leadership

Communication

Databricks

PySpark

Spark Structured Streaming

Delta Lake

Python

SQL

Azure

Observability

Tools

Databricks platform

Azure Data Platform

Azure DevOps

Job description

EPAM Systems in London, hybrid role, seeks a Lead Databricks Engineer to design and lead enterprise-scale Azure and Databricks data platforms for advanced analytics and real-time processing. You will own platform architecture, governance, cost optimization, and ensure delivery of high-quality solutions in a complex data-driven environment.

You will lead architecture, enforce standards, optimize pipelines, mentor developers, and collaborate with senior stakeholders to align platform initiatives