Databricks Lead Engineer

NTT; Greater London; On-site; GBP 70,000 - 100,000; Full time

Job summary

NTT is seeking a Senior Data Engineer to lead and deliver scalable Databricks-based data pipelines across cloud platforms. You will design, implement and optimise lakehouse architectures, drive governance and collaborate with analytics leadership to meet client requirements.

The role requires 5–8 years in data engineering with 3+ years on Databricks, strong Python/SQL skills, and cloud experience. Hybrid working in Greater London is common for client engagements.

Qualifications

• Experience leading or mentoring teams of engineers to deliver high-quality scalable data solutions.
• Hands-on expertise across the data lifecycle: ingestion, transformation, modelling, governance, and consumption.
• Strong problem-solving, analytical, and communication skills.
• Proven experience in data engineering and pipeline development on Databricks and cloud-native platforms.
• Exposure to AI/ML workloads desirable.

Responsibilities

• Lead data engineering projects delivering high-performing, scalable data pipelines.
• Collaborate with client stakeholders up to Head of Data Engineering and Analytics leadership.
• Mentor teams of engineers to ensure best practices and quality.

Skills

Databricks

Spark

Python

SQL

Delta Lake

Data Modelling

Data Warehousing

CI/CD

GitHub/Bitbucket

Cloud Platforms

DBT

Matillion

Talend

Pyspark/Scala

ML/AI workloads

Tools

Unity Catalog

MLflow

Job description

• Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive
• Client Engagement & Delivery
• Data Pipeline Development (Batch and Streaming)
• Databricks & Lakehouse Architectures
• Data Modelling & Optimisation (Delta Lake, Medallion architecture)
• Collaboration & Best Practices
• Quality, Governance & Security
• Business Relationships:
• Solution Architects
• Data Engineers, Developers, ML Engineers, and Analysts
• Client stakeholders up to Head of Data Engineering, Chief Data Architect, and Analytics leadership
• Measures of Success:
• Delivery of high-performing, scalable, and secure data pipelines aligned to client requirements
• High client satisfaction and successful adoption of Databricks-based solutions
• Demonstrated ability to innovate and improve data engineering practices
• Contribution to the growth of the practice through reusable assets, accelerators, and technical leadership

Experience leading or mentoring teams of engineers to deliver high-quality scalable data solutionsHands-on expertise across the data lifecycle: ingestion, transformation, modelling, governance, and consumptionStrong problem-solving, analytical, and communication skillsProven experience in data engineering and pipeline development on Databricks and cloud-native platformsStrong consulting values with ability to collaborate effectively in client-facing environmentsExposure to AI/ML workloads desirableUnderstanding of data governance, security, and compliance frameworksFamiliarity with Databricks Workflows and other orchestration toolsFamiliarity with medallion architectures, data lakehouse principles and distributed data processingDeep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow)Experience with version control tools (GitHub, Bitbucket) and CI/CD pipelinesStrong SQL and Python (or equivalent language) skills for data manipulation and automationHands-on experience with cloud platforms (AWS, Azure, GCP)Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalentKnowledge of data modelling methodologies (star schemas, Data Vault, Kimball, Inmon)Preferred: BSc/MSc in Computer Science, Data Engineering, or related fieldDatabricks certifications (Data Engineer Professional) highly desirableEducation: University degree requiredExperience: Minimum 5–8 years in data engineering, data warehousing, or data architecture roles, with at least 3+ years working with Databricks