Senior Azure Databricks Architect — Hybrid London

EPAM Systems Ltd; Greater London; Hybrid; GBP 110,000 - 150,000; Full time

Benefits offered by this job

ESPP

Private medical

Dental care

Life insurance

Income protection

Pension plan

Cyclescheme

Techscheme

Season ticket loan

Lunch in-office

In-house training

Certifications

Discretionary bonus

Long-Term Incentive

Job summary

EPAM Systems Ltd. in London is seeking a Lead Databricks Engineer to design and lead enterprise-scale Azure and Databricks data platforms, enabling advanced analytics and real-time processing.

You will own platform architecture, governance, cost optimization and deliver high-quality solutions in a complex, data-driven environment, mentoring teams and guiding cross-functional stakeholders.

Qualifications

• 8+ years in data engineering, including 3+ years in a tech lead role.
• Hands-on Databricks for enterprise-scale workloads.
• Proficient in PySpark, Spark Structured Streaming, Delta Lake; Python and SQL performance optimization.

Responsibilities

• Lead architecture design and implementation of scalable solutions on Azure Databricks.
• Define platform standards, ensuring compliance with data governance and security policies.
• Build and optimize Databricks streaming workloads, including Structured Streaming and Delta Live Tables.
• Collaborate with cross-functional teams to translate requirements into robust technical solutions.
• Drive platform performance tuning, cost optimization and monitoring strategies for cloud workloads.
• Provide leadership across the full engineering life cycle, including solution delivery and production support.
• Mentor and guide developers, establishing best practices for data engineering on Azure and Databricks.
• Participate in technical decision-making and review designs for scalability and maintainability.
• Implement observability for critical pipelines and maintain quality assurance standards.
• Engage directly with stakeholders to ensure alignment on strategic platform initiatives.

Skills

Databricks

PySpark

Structured Streaming

Delta Lake

Python

SQL optimization

Azure Data Platform

Data governance

Observability

Leadership

Job description

EPAM Systems Ltd. in London is seeking a Lead Databricks Engineer to design and lead enterprise-scale Azure and Databricks data platforms, enabling advanced analytics and real-time processing.

You will own platform architecture, governance, cost optimization and deliver high-quality solutions in a complex, data-driven environment, mentoring teams and guiding cross-functional stakeholders.