Data Engineer

Mayfair IT Consultancy Ltd

Greater London

On-site

GBP 60,000 - 120,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Mayfair IT Consultancy Ltd is recruiting Data Engineers for a major UK Government data programme. You will design, build and maintain robust data pipelines using Python and SQL to support large-scale data processing and quality control.

In this role you will transform datasets, ensure data quality, and deliver reusable engineering solutions while applying Git, automated testing and CI/CD practices. SC Security Clearance is required.

Qualifications

  • Strong practical Python programming experience.
  • SQL experience for data engineering and transformation.
  • Experience building, testing and maintaining data pipelines or complex scripts.
  • Data cleansing, preparation, transformation and quality validation.
  • Git, version control and collaborative engineering practices.
  • Code reviews, automated testing and technical documentation.
  • Ability to investigate, resolve and prevent technical issues.
  • Strong problem-solving and communication skills.
  • Ability to work independently and within multidisciplinary teams.

Responsibilities

  • Design, develop and maintain data pipelines, frameworks and systems.
  • Write clear, reliable and well-tested Python code.
  • Use SQL to prepare, clean, transform and validate data at scale.
  • Investigate data quality issues and resolve technical problems.
  • Develop reusable components and improve engineering processes.
  • Apply Git, code reviews, automated testing and CI/CD practices.
  • Document solutions and follow agreed engineering standards.
  • Collaborate with technical and non-technical stakeholders.
  • Identify opportunities for automation and continuous improvement.

Skills

Strong problem-solving
Communication
Independence
Team collaboration
Attention to detail

Tools

Python
SQL
ETL/ELT pipeline development
PySpark
Cloudera
GCP
CI/CD
Git
Automated testing

Job description

Overview

We are seeking experienced Data Engineers to join a major UK Government data programme, developing and maintaining reliable data pipelines and engineering solutions.

You will use Python and SQL to build, test and improve data processes, prepare and transform datasets, maintain data quality and deliver high-quality, reusable engineering solutions.

Key Responsibilities
  • Design, develop and maintain data pipelines, frameworks and systems.
  • Write clear, reliable and well-tested Python code.
  • Use SQL to prepare, clean, transform and validate data at scale.
  • Investigate data quality issues and resolve technical problems.
  • Develop reusable components and improve engineering processes.
  • Apply Git, code reviews, automated testing and CI/CD practices.
  • Document solutions and follow agreed engineering standards.
  • Collaborate with technical and non-technical stakeholders.
  • Identify opportunities for automation and continuous improvement.
Essential Skills & Experience
  • Strong practical Python programming experience.
  • SQL experience for data engineering and transformation.
  • Experience building, testing and maintaining data pipelines or complex scripts.
  • Data cleansing, preparation, transformation and quality validation.
  • Git, version control and collaborative engineering practices.
  • Code reviews, automated testing and technical documentation.
  • Ability to investigate, resolve and prevent technical issues.
  • Strong problem-solving and communication skills.
  • Ability to work independently and within multidisciplinary teams.
Technical Skills & Technologies
  • Python and SQL
  • ETL / ELT pipeline development
  • PySpark and large-scale data processing
  • Cloudera or comparable enterprise data platforms
  • GCP or equivalent cloud environments
  • Data profiling, cleansing and quality management
  • Reusable Python packages and engineering frameworks
  • CI/CD, Git and automated testing
  • Data integration and pipeline optimisation

Comparable enterprise technologies and transferable experience are welcome.

Seniority & Delivery Expectations
  • Develop reliable, scalable and maintainable data solutions.
  • Produce high-quality, tested code.
  • Apply sound technical judgement and resolve problems proactively.
  • Follow engineering standards and contribute to improvements.
  • Communicate effectively and share knowledge.
Desirable
  • UK Government or public-sector experience.
  • Experience with large-scale or complex datasets.
  • Strong PySpark, Cloudera or GCP experience.
  • Reusable Python package development.
  • Data governance, security and compliance knowledge.
Clearance Requirement

SC Security Clearance is required for this assignment.

Candidates should be able to confirm their current security clearance status and meet the required screening and onboarding requirements.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Modular Data • United Kingdom

Remote
GBP 70,000 - 110,000
Fully remote (UK-based)
Senior Data Engineer
Senior Data Engineer

Ceox • United Kingdom

Remote
GBP 83,000 - 131,000
Data Engineer (contract)
Data Engineer (contract)

Modular Data • United Kingdom

Remote
GBP 101,000 - 166,000
Outside IR35 contract
Fully remote (UK-based)
Meaningful public sector projects
+1
Senior Data Engineer
Senior Data Engineer

ByteHire Limited. • United Kingdom

Remote
GBP 90,000 - 130,000
Data Engineer
Data Engineer

Datacareers • City Of London

On-site
GBP 70,000 - 110,000
Senior Data Engineer
Senior Data Engineer

NPAworldwide • Manchester

Hybrid
GBP 50,000 - 70,000
Flexible working
Hybrid/remote working
Learning & Development
+5
Data Engineer
Data Engineer

G Digital • Greater London

Hybrid
GBP 36,000 - 60,000
Employer pension 5% match
Training budget
Hybrid work model
Data Platform Engineer
Data Platform Engineer

NPAworldwide • Leeds

Hybrid
GBP 50,000 - 70,000
Flexible working
Hybrid/remote working
Learning & Development
+5
Data Solutions Engineer
Data Solutions Engineer

NPAworldwide • Cardiff

Hybrid
GBP 50,000 - 70,000
Flexible working
Hybrid/remote working
Dev budget for conferences and courses
+3
Big Data Engineer
Big Data Engineer

NPAworldwide • Liverpool

Hybrid
GBP 50,000 - 70,000
Flexible working
Hybrid/remote working
Learning & Development
+5