Data Engineer

eTeam; Greater London; Hybrid; GBP 56,000 - 76,000; Full time

Job summary

eTeam is seeking a data engineer to build and maintain data pipelines and infrastructure in London. The role involves hands-on work with Azure Databricks, Spark, and other big data tools to store and process data efficiently.

You will collaborate on design, development, and optimization of data workflows, ensuring reliability, scalability, and quality across the data platform and cloud environments.

Qualifications

• Experience designing data pipelines and ETL workflows.
• Strong knowledge of Azure Databricks and Spark.
• Proficient in Python and cloud technologies.

Responsibilities

• Design, develop, and optimize data pipelines.
• Ensure data reliability, scalability, and quality.
• Monitor and troubleshoot data workflows.

Skills

Azure Databricks

Spark

Big data tools

Python

ETL processes

Cloud technologies

CI/CD

Tools

Azure Databricks

Spark

CI/CD tooling

Job description

We are a Global Recruitment specialist that provides support to the clients across EMEA, APAC, US and Canada. We have an excellent job opportunity for you.

Location: London

Pay Rate: £358 per day all inc. (PAYE through Umbrella)

Sector: Insurance

Days on site: 2-3 days/week onsite

Role Description:

Objective:

Build and maintain data pipelines and infrastructure to support data storage and processing.

Key Responsibilities:

• Design, develop, and optimize data pipelines using Azure Databricks.
• Ensure data reliability, scalability, and quality.
• Monitor and troubleshoot data workflows.

Skills Required:

• Expertise in Azure Databricks, Spark, and big data tools.
• Knowledge of Python, ETL processes and cloud technologies.
• Familiarity with CI/CD practices for data pipelines.