Data Engineer - Databricks | AWS |Snowflake

Korn Ferry; West Midlands; Hybrid; GBP 90,000 - 120,000; Full time

Job summary

Korn Ferry is seeking an experienced Senior Data Engineer to join a major enterprise technology programme in the financial services sector. The role focuses on designing, building and optimising large-scale data pipelines in an AWS environment, with support for AI/ML initiatives and generative AI data platforms.

The candidate will design and implement ETL/ELT workflows, work on data lakehouse architectures, and ensure data quality, governance, security, and monitoring across production systems.

Qualifications

• 5+ years of commercial Data Engineering experience.
• Strong programming experience with Java and/or Scala.
• Strong AWS experience, ideally including technologies such as S3, EMR, Glue and IAM.
• Advanced SQL development skills.
• Experience designing and developing ETL/ELT pipelines.
• Experience with large-scale distributed data-processing environments.
• Databricks and/or Snowflake.

Responsibilities

• Design, build and maintain high-throughput data pipelines using Apache Spark and cloud-native technologies.
• Develop scalable ETL/ELT workflows for structured and unstructured datasets.
• Build and optimise data-processing solutions within AWS environments.
• Develop performant solutions using Java or Scala alongside advanced SQL.
• Support modern data lake and lakehouse architectures.
• Work alongside AI/ML teams to provide data platforms supporting Generative AI, machine learning and advanced analytics use cases.
• Implement appropriate data quality, governance, security and monitoring standards.
• Troubleshoot complex data-processing and distributed-system issues within production environments.

Skills

Java/Scala programming

SQL development

Data pipelines design

Distributed data processing

Tools

Apache Spark

AWS (S3, EMR, Glue, IAM)

Databricks

Snowflake

Job description

Working Pattern: Hybrid - 3 days per week onsite at Birmingham office

We are looking for an experienced Senior Data Engineer to join a major enterprise technology programme within the financial services sector. You will be responsible for designing, building and optimising large-scale data pipelines and distributed processing solutions within an AWS environment. The role will also support modern AI/ML and Generative AI initiatives by developing scalable, secure and reliable data platforms.

Key Responsibilities

• Design, build and maintain high-throughput data pipelines using Apache Spark and cloud-native technologies.
• Develop scalable ETL/ELT workflows for structured and unstructured datasets.
• Build and optimise data-processing solutions within AWS environments.
• Develop performant solutions using Java or Scala alongside advanced SQL.
• Support modern data lake and lakehouse architectures.
• Work alongside AI/ML teams to provide data platforms supporting Generative AI, machine learning and advanced analytics use cases.
• Implement appropriate data quality, governance, security and monitoring standards.
• Troubleshoot complex data-processing and distributed-system issues within production environments.
• 5+ years of commercial Data Engineering experience.
• Strong programming experience with Java and/or Scala.
• Strong AWS experience, ideally including technologies such as S3, EMR, Glue and IAM.
• Advanced SQL development skills.
• Strong experience designing and developing ETL/ELT pipelines.
• Experience working with large-scale and distributed data-processing environments.
• Databricks and/or Snowflake.

About Korn Ferry

Korn Ferry unleashes potential in people, teams, and organizations. We work with our clients to design optimal organization structures, roles, and responsibilities. We help them hire the right people and advise them on how to reward and motivate their workforce while developing professionals as they navigate and advance their careers. To learn more, please visit Korn Ferry at www.Kornferry.com