Databricks Data Engineer

NTT; Greater London; On-site; GBP 70,000 - 110,000; Full time

Job summary

NTT in London seeks a Databricks Data Engineer to design and implement scalable data pipelines, Lakehouse architectures, and governance-enabled data products for clients.

You will lead batch and streaming ETL/ELT, collaborate with solution architects and data teams, and mentor engineers while delivering high-quality, secure data solutions.

Qualifications

• 5–8 years in data engineering, warehousing, or data architecture.
• 3+ years working with Databricks, Spark, PySpark or Scala.
• Strong SQL and Python skills for data manipulation.
• Experience with cloud platforms (AWS, Azure, GCP).
• Knowledge of data governance, security and compliance.
• Familiarity with medallion architectures and lakehouse concepts.

Responsibilities

• Client engagement and delivery of data solutions for clients.
• Data pipeline development (batch and streaming) on Databricks.
• Designing lakehouse architectures and Delta Lake modelling.
• Collaborating with architects, engineers, ML teams and analysts.
• Ensuring governance, security and data quality across pipelines.
• Mentoring engineers and contributing to reusable accelerators.

Skills

Databricks

SQL

Python

ETL/ELT

Delta Lake

Data Modeling

CI/CD

GitHub

Cloud Platforms

Security & Governance

Education

BSc in Computer Science

Databricks certification

Tools

DBT

Matillion

Talend

Databricks Workflows

Job description

• We are seeking a highly skilled Databricks Data Engineer to join our Data & AI practice.
• The successful candidate will have deep expertise in building scalable data pipelines, optimizing Lakehouse architectures and enabling advanced analytics and AI use cases on the Databricks platform.
• This role is critical in building and optimizing modern data ecosystems that enable data-driven decision making, advanced analytics, and AI capabilities for our clients
• As a trusted practitioner, you will design and implement robust ETL/ELT workflows, integrate real-time and batch data sources, and enable secure, well-governed data products and pipelines.
• You will thrive in a collaborative, client-facing environment, with a passion for solving complex data challenges, driving innovation and ensuring the seamless delivery of data solutions

Primary Responsibilities;

• Client Engagement & Delivery
• Data Pipeline Development (Batch and Streaming)
• Databricks & Lakehouse Architectures
• Data Modelling & Optimisation (Delta Lake, Medallion architecture)
• Collaboration & Best Practices
• Quality, Governance & Security

Business Relationships;

• Solution Architects
• Data Engineers, Developers, ML Engineers, and Analysts
• Client stakeholders up to Head of Data Engineering, Chief Data Architect, and Analytics leadership

Measures of Success;

• Delivery of high-performing, scalable, and secure data pipelines aligned to client requirements
• High client satisfaction and successful adoption of Databricks-based solutions
• Demonstrated ability to innovate and improve data engineering practices
• Contribution to the growth of the practice through reusable assets, accelerators, and technical leadership

Strong consulting values with ability to collaborate effectively in client-facing environmentsStrong problem-solving, analytical, and communication skillsHands-on expertise across the data lifecycle: ingestion, transformation, modelling, governance, and consumptionExperience leading or mentoring teams of engineers to deliver high-quality scalable data solutionsProven experience in data engineering and pipeline development on Databricks and cloud-native platformsUnderstanding of data governance, security, and compliance frameworksProficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalentHands-on experience with cloud platforms (AWS, Azure, GCP)Strong SQL and Python (or equivalent language) skills for data manipulation and automationFamiliarity with Databricks Workflows and other orchestration toolsDeep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow)Experience with version control tools (GitHub, Bitbucket) and CI/CD pipelinesExposure to AI/ML workloads desirableFamiliarity with medallion architectures, data lakehouse principles and distributed data processingKnowledge of data modelling methodologies (star schemas, Data Vault, Kimball, Inmon)Experience: Minimum 5–8 years in data engineering, data warehousing, or data architecture roles, with at least 3+ years working with DatabricksPreferred: BSc/MSc in Computer Science, Data Engineering, or related fieldDatabricks certifications (Data Engineer Professional) highly desirableEducation: University degree required