Senior Data Engineer | Python | Azure Data Factory | Kafka | Postgresql | Elasticsearch | Active SC

Methods Business and Digital Technology Limited; Slimbridge; On-site; GBP 75,000 - 85,000; Full time

Benefits offered by this job

Career development

Certification support

Modern cloud tech

Significant programmes

Clearance progression

Growing company

Job summary

Methods Business and Digital Technology Limited is seeking a Senior Data Engineer to design, build and maintain scalable data platforms for secure analytics and operational decision-making. You will work across the full data lifecycle, combining on-premises infrastructure with Azure cloud technologies.

The role requires strong Python development, ETL/ELT expertise, and experience with data security and access governance.

Qualifications

• Strong Python development experience in enterprise environments.
• Proven ETL/ELT design and delivery capabilities.
• Expertise with Azure Data Factory and cloud data services.
• Active SC clearance with willingness to pursue DV clearance.

Responsibilities

• Design, build and maintain scalable data pipelines across on-premises and Azure.
• Develop ETL/ELT processes for structured, semi-structured and unstructured data.
• Implement hybrid data solutions across on-premises and cloud environments.
• Manage PostgreSQL database deployments and development.
• Operate Elasticsearch deployments and optimize performance.
• Support real-time streaming using Kafka and NATS.
• Contribute to security, access controls and platform governance.

Skills

Python development

ETL/ELT pipelines

Azure Data Factory

Kafka/NATS

PostgreSQL

Elasticsearch

Data security & governance

Hybrid cloud/on-prem

Active SC clearance

Job description

Senior Data Engineer

Ledbury/Malvern/Gloucester/Cheltenham Area
On-site - 5 Days Per Week
Up to £85,000 + Benefits
Security Clearance: Active SC Clearance Required

Methods Business and Digital Technology is a £100M+ IT Services Consultancy delivering complex digital transformation programmes across government and regulated sectors. We are seeking an experienced Senior Data Engineer to join a growing team supporting the delivery of secure, mission‑critical data platforms.

This role requires an individual who is comfortable operating across hybrid environments, combining on‑premises infrastructure with Azure cloud technologies to deliver scalable, secure and high‑performing data solutions.

The Role

As a Senior Data Engineer, you will design, build and maintain sophisticated data infrastructure supporting analytics, reporting and operational decision‑making. You will work across the full data life cycle, developing robust data pipelines, implementing automation and supporting Real Time data streaming capabilities.

Responsibilities

• Design, develop and maintain scalable data pipelines using Python, Go and Azure Data Factory
• Develop ETL and ELT processes supporting structured, semi‑structured and unstructured data
• Implement and manage hybrid data solutions across on‑premises and Azure environments
• Build and maintain PostgreSQL database solutions
• Manage Elasticsearch deployments and optimisation
• Implement workflow automation using Azure Data Factory
• Develop and support Real Time event streaming solutions using Kafka and NATS
• Support security, authentication and access management using technologies such as Keycloak
• Collaborate with technical teams to deliver resilient, secure and scalable data platforms

Skills

• Strong commercial Python development experience
• Experience developing ETL/ELT solutions within enterprise environments
• Azure Data Factory expertise
• Experience with Kafka and/or NATS event streaming technologies
• PostgreSQL administration and development experience
• Elasticsearch experience
• Strong understanding of data security, access controls and platform governance
• Experience operating within hybrid cloud and on‑premises environments
• Active SC Clearance

Desirable

• Azure certifications
• Experience with Go development
• DevOps and automation experience
• Experience supporting highly secure government or defence environments
• Willingness and eligibility to progress to DV Clearance

Benefits

• Up to £85,000 salary
• Work on nationally significant programmes
• Join a growing and highly respected consultancy
• Exposure to modern cloud, data and platform technologies
• Career development and certification support
• Opportunity to progress through higher levels of security clearance

Active SC Clearance is mandatory for this position. Candidates must also be willing to undergo DV clearance as part of future project requirements.