Data Engineer (Databricks)

Korn Ferry; City of Westminster; Hybrid; GBP 95,000 - 120,000; Full time

Job summary

Korn Ferry is seeking a Data Software Engineer - Azure Databricks to join our Financial Services client in London. This hybrid role focuses on designing and building data engineering solutions using Azure Databricks and related Azure services to process large-scale datasets.

You will collaborate with cross-functional teams to deliver optimized, reliable data pipelines, ensure code quality, and participate in Agile development. 6+ years of experience with Azure technologies is expected.

Qualifications

• 6+ years of data software engineering with a focus on Azure technologies.
• Expertise in Azure Databricks development for large datasets.
• Strong knowledge of Azure Data Factory and ETL workflows.
• Experience in Agile with cross-functional collaboration.
• Ability to optimize data pipelines and distributed processing.

Responsibilities

• Develop robust, scalable data engineering solutions using Azure Databricks and Azure Data Factory.
• Design and optimize data pipelines for large-scale datasets with high performance.
• Participate in customer discussions, requirement analysis, and technical solution design.
• Ensure code quality, performance tuning, and adherence to best practices.
• Communicate project status and escalate critical issues to stakeholders.
• Collaborate with team members to deliver data-driven solutions in an Agile environment.
• Expertise in Azure Databricks development for large datasets.
• Strong working knowledge of Azure Data Factory for building ETL workflows.
• Experience with performance optimization of data pipelines and distributed processing setups.
• Solid understanding of Azure services relevant to data engineering and cloud architecture.
• Strong problem-solving ability with effective communication skills for team collaboration.

Skills

Azure Databricks

Azure Data Factory

Data engineering

ETL workflows

Distributed processing

Cloud architecture

Agile

CI/CD

Data security

Tools

Azure Databricks development

Azure Data Factory development

CI/CD tooling

Job description

We’re looking for a Data Software Engineer - Azure Databricks to join our client (Financial Services) team in London, UK in an hybrid working mode. In this role, you will design and build data engineering solutions leveraging Azure Databricks and related Azure services, enabling efficient processing and transformation of large-scale datasets. You will collaborate with cross-functional teams to ensure the delivery of optimized, reliable, and scalable data pipelines aligned with business requirements.

Responsibilities

• Develop robust, scalable data engineering solutions using Azure Databricks and Azure Data Factory
• Design and optimize data pipelines for processing large volumes of data (multi-million records) with high performance
• Participate in customer discussions, requirement analysis, and technical solution design
• Ensure code quality, performance tuning, and adherence to best practices throughout development
• Communicate project status effectively and escalated critical issues to stakeholders and management
• Actively collaborate with team members to deliver data-driven solutions in an Agile environment
  6+years of experience in data software engineering with a strong focus on Azure technologies
• Expertise in Azure Databricks development, including efficient handling of large datasets
• Strong working knowledge of Azure Data Factory for building complex ETL workflows
• Experience with performance optimization of data pipelines and distributed processing setups
• Solid understanding of Azure services relevant to data engineering and cloud architecture
• Strong problem-solving ability with effective communication skills for team collaboration

Nice to Have Skills

• Experience working in Agile teams, including Scrum or Kanban methodologies
• Familiarity with CI/CD practices and tools for data engineering workflows
• Databricks certification or equivalent credentials
• Understanding of data security, compliance, and governance best practices in cloud environments