Data Engineer

TN United Kingdom; Southend-on-Sea; On-site; GBP 50,000 - 80,000; Full time

Job summary

An innovative firm is seeking a skilled Data Engineer to design and maintain scalable data pipelines. This role involves collaborating with various teams to ensure data integrity and security while optimizing workflows in cloud environments. The ideal candidate will have extensive experience in Databricks and Apache Spark, with a strong understanding of modern data architectures. This is an exciting opportunity to contribute to cutting-edge data solutions in a hybrid work environment. If you're passionate about data engineering and ready to make an impact, we want to hear from you!

Qualifications

• 5+ years of Data Engineering experience required.
• Proven expertise in Databricks and Apache Spark is essential.

Responsibilities

• Design and maintain scalable data pipelines using Databricks.
• Collaborate with teams to deliver clean, reliable datasets.
• Implement best practices in data governance and security.

Skills

Data Engineering

Databricks

Apache Spark

SQL

Python

AWS

Azure

GCP

Data Architectures

CI/CD Pipelines

Terraform

Airflow

MLflow

Data Warehousing

Consulting

Communication Skills

Job description

Social network you want to login/join with:

Position: Data Engineer

Employment Type: Contract, Full time

Start: ASAP

Location: London - Hybrid

Languages: English

Key Skills:

• 5+ years of Data Engineering experience
• Proven expertise in Databricks (including Delta Lake, Workflows, Unity Catalog)
• Strong command of Apache Spark, SQL, and Python
• Hands-on experience with cloud platforms (AWS, Azure, or GCP)
• Understanding of modern data architectures (e.g., Lakehouse, ELT/ETL pipelines)
• Familiarity with CI/CD pipelines and infrastructure-as-code tools (Terraform is a plus)
• Experience with Airflow or similar orchestration tools
• Familiarity with MLflow or MLOps practices
• Knowledge of data warehousing solutions (Snowflake, Redshift, BigQuery)
• Consulting background is a plus
• Strong communication skills (oral & written)
• Rights to work in the UK is a must (No Sponsorship available)

Responsibilities:

• Design, build, and maintain scalable and efficient data pipelines using Databricks and Apache Spark
• Collaborate with Data Scientists, Analysts, and Product teams to understand data needs and deliver clean, reliable datasets
• Optimize data workflows and storage (Delta Lake, Lakehouse architecture)
• Manage and monitor data pipelines in cloud environments (AWS, Azure, or GCP)
• Work with structured and unstructured data across multiple sources
• Implement best practices in data governance, data security, and data quality
• Automate workflows and data validation tasks using Python, SQL, and Databricks notebooks

If you are interested in this position and would like to discuss further, please apply with your latest CV or share it directly with me at [emailprotected].