Databricks Engineer

Syntax Consultancy Ltd; City Of London; Hybrid; GBP 80,000 - 100,000; Full time

Job summary

A leading global IT and Digital transformation consultancy is seeking a Databricks Engineer with active SC Security Clearance for a 6-month contract in Central London (Hybrid). The role involves developing a cutting-edge Azure Databricks platform focused on economic data modelling and analysis. Candidates must have strong expertise in data engineering, hands-on experience with Azure services, and the ability to manage complex data pipelines. The position offers a flexible working model, combining remote and on-site work.

Qualifications

• Strong hands-on Azure Databricks expertise is a must.
• Experience of working with complex data pipelines and large data sets.
• Good knowledge of data quality and governance processes.

Responsibilities

• Designing, developing, and optimising data pipelines.
• Implementing complex data transformations using Spark and PySpark.
• Ensuring data quality and reliability in data processes.

Skills

In-depth Data Engineering

Azure Databricks expertise

Data modelling techniques

Data pipeline optimization

Experience with SQL/MySQL databases

Education

Azure Data Engineer certification

Tools

Azure Data Services

Azure Data Factory

Azure Blob Storage

Power BI

Tableau

Job description

Databricks Engineer (SC Cleared)

London (Hybrid)

6 Month Contract

£(Apply online only)/day (Inside IR35)

Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid).

Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start asap in Dec 2025 / Jan 2026.

Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the Central London office.

A chance to work with a leading global IT and Digital transformation business specialising in Government projects:

• In-depth Data Engineering + strong hands-on Azure Databricks expertise.
• Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database.
• Designing, developing, building + optimising data pipelines, implementing data transformations, ensuring data quality and reliability.
• Deep Data Warehousing knowledge including data modelling techniques + data integration patterns.
• Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design.
• Implementing complex data transformations using Spark, PySpark or Scala + working with SQL / MySQL databases.
• Experience with data quality, data governance processes, Git version control + Agile development environments.
• Azure Data Engineer certification preferred -eg- Azure Data Engineer Associate.
• Advantageous skills: Azure Event Hubs, Kafka, data visualisation tools, Power BI, Tableau, Azure DevOps, Docker, Kubernetes, Jenkins.