Senior Databricks Data Engineer — Big Data Pipelines

Korn Ferry; Greater London; On-site; GBP 83,025 - 101,475; Full time

Job summary

Korn Ferry is seeking a Senior Data Engineer specializing in Databricks and Big Data Pipelines. This UK remote position requires active Security Check (SC) clearance. The role involves designing and optimizing data pipelines on Databricks while ensuring scalability and compliance. Strong expertise in Databricks, Python, and advanced SQL is essential.

You will lead the pipeline lifecycle and collaborate with various teams. The role offers a rate of £450-550 per day inside IR35.

Qualifications

• Active SC Clearance.
• Expertise in Databricks, including Delta Lake and Unity Catalog.
• Strong background in Python, PySpark and advanced SQL.
• Track record of building high-performance, large-scale data solutions.

Responsibilities

• Design, develop and optimize data pipelines using Databricks.
• Lead the full pipeline lifecycle: analysis, architecture, development.
• Implement ingestion and standardization workflows for diverse data sources.
• Collaborate with engineering and product teams during project delivery.
• Promote data platform best practices.

Skills

Databricks

Python

PySpark

SQL

AWS

Job description

Korn Ferry is seeking a Senior Data Engineer specializing in Databricks and Big Data Pipelines. This UK remote position requires active Security Check (SC) clearance. The role involves designing and optimizing data pipelines on Databricks while ensuring scalability and compliance. Strong expertise in Databricks, Python, and advanced SQL is essential.

You will lead the pipeline lifecycle and collaborate with various teams. The role offers a rate of £450-550 per day inside IR35.