Data Engineer

TechYard; Manchester; On-site; GBP 70,000 - 90,000; Full time

Job summary

TechYard, a specialist data and AI consultancy in the UK, is growing its Databricks capability and seeks a Senior Databricks Consultant to join the delivery team.

You’ll work on modern cloud data platforms and Lakehouse programmes, helping enterprise clients design, build and optimise scalable data engineering solutions.

Qualifications

• Strong commercial experience in Data Engineering.
• Hands-on experience with Databricks and/or Apache Spark.
• Strong PySpark and SQL development skills.
• Experience designing, building and supporting data pipelines.
• Experience working with Azure or AWS.
• Good understanding of modern data platforms and Lakehouse architectures.
• Strong analytical and problem-solving abilities.
• Excellent communication skills, with the ability to work effectively with both technical and non-technical stakeholders.

Responsibilities

• Design, develop and maintain data pipelines and data engineering solutions using Databrick(s).
• Build scalable data processing solutions using PySpark and SQL.
• Work across Databricks Lakehouse environments, supporting implementation, optimisation and ongoing improvement.
• Develop and optimise batch and data processing workflows.
• Contribute to technical solution design and data platform architecture.
• Work closely with technical and business stakeholders to understand requirements and deliver effective solutions.
• Support testing, deployment and production implementation activities.
• Troubleshoot performance, reliability and data quality issues.
• Follow and contribute to engineering best practices across Databricks and cloud data platforms.
• Support clients through different stages of their data platform and lakehouse transformation journeys.

Skills

Data engineering

Databricks

PySpark

SQL

Azure

AWS

Data pipelines

Communication

Problem solving

Tools

Databricks

Apache Spark

Job description

A specialist data and AI consultancy is growing its Databricks capability and is looking for a Senior Databricks Consultant to join its delivery team.

This is an opportunity to work on modern cloud data platforms and Databricks Lakehouse programmes, helping enterprise clients design, build and optimise scalable data engineering solutions.

You’ll work closely with experienced data architects, engineers and client stakeholders, with Databricks at the centre of the role. The position would suit someone who enjoys hands‑on engineering while also contributing to technical solution design and client delivery.

What You'll Be Doing

• Design, develop and maintain data pipelines and data engineering solutions using Databrick
• sBuild scalable data processing solutions using PySpark and SQL
• Work across Databricks Lakehouse environments, supporting implementation, optimisation and ongoing improvement
• Develop and optimise batch and data processing workflows
• Contribute to technical solution design and data platform architecture
• Work closely with technical and business stakeholders to understand requirements and deliver effective solutions
• Support testing, deployment and production implementation activities
• Troubleshoot performance, reliability and data quality issues
• Follow and contribute to engineering best practices across Databricks and cloud data platforms
• Support clients through different stages of their data platform and lakehouse transformation journeys

What We're Looking For

• Strong commercial experience in Data Engineering
• Hands‑on experience with Databricks and/or Apache Spark
• Strong PySpark and SQL development skills
• Experience designing, building and supporting data pipelines
• Experience working with Azure or AWS
• Good understanding of modern data platforms and Lakehouse architectures
• Strong analytical and problem‑solving abilities
• Excellent communication skills, with the ability to work effectively with both technical and non‑technical stakeholders

Experience with any of the following would be highly beneficial

• Unity Catalog
• MLfow
• Databricks optimisation and performance tuning
• AI, Machine Learning or Generative AI initiatives
• AI‑assisted development tools such as Claude, GitHub Copilot or Cursor
• Consultancy, professional services or client‑facing delivery experience