Senior Data Engineer | Python | Azure Data Factory | Kafka | Postgresql | Elasticsearch | Active SC

Methods Business and Digital Technology Limited; Gloucester; On-site; GBP 51,000 - 85,000; Full time

Job summary

Methods Business and Digital Technology Limited in Ledbury/Malvern/Gloucester/Cheltenham Area is seeking an experienced Senior Data Engineer to deliver secure, scalable data platforms. The role is on-site with a path to hybrid work, offering up to £85,000 plus benefits.

Active SC clearance is required, with DV potential in the future. You will design, build and maintain data infrastructure spanning analytics, reporting and operational decision-making, across on-premises and Azure cloud

Qualifications

• Experience designing and building data pipelines for analytics and decision-making.
• Proficient in ETL/ELT processes across structured, semi-structured and unstructured data.
• Strong Python development experience in enterprise environments.
• Experience with Azure Data Factory for data integration pipelines.
• Knowledge of Kafka and/or NATS for real-time streaming.
• PostgreSQL administration and development experience.
• Understanding of data security, access controls and governance.
• Experience operating in hybrid cloud and on-premises environments.
• Active SC Clearance is required for this role.

Responsibilities

• Design, build and maintain scalable data pipelines using Python, Go and Azure Data Factory.
• Develop ETL/ELT processes for varied data formats.
• Implement and manage hybrid data solutions across on-premises and Azure environments.
• Build and maintain PostgreSQL database solutions.
• Manage Elasticsearch deployments and optimisation.
• Implement workflow automation using Azure Data Factory.
• Develop and support Real Time event streaming with Kafka and NATS.
• Support security, authentication and access management (e.g., Keycloak).
• Collaborate with technical teams to deliver resilient, secure data platforms.

Skills

Python

ETL/ELT

Azure Data Factory

Kafka/NATS

PostgreSQL

Elasticsearch

Security/Governance

Hybrid/On-prem

SC Clearance

Job description

Senior Data Engineer

Ledbury/Malvern/Gloucester/Cheltenham Area
On-site - 5 Days Per Week
Up to £85,000 + Benefits
Security Clearance: Active SC Clearance Required

Methods Business and Digital Technology is a £100M+ IT Services Consultancy delivering complex digital transformation programmes across government and regulated sectors. We are seeking an experienced Senior Data Engineer to join a growing team supporting the delivery of secure, mission-critical data platforms.

This role requires an individual who is comfortable operating across hybrid environments, combining on-premises infrastructure with Azure cloud technologies to deliver scalable, secure and high-performing data solutions.

The Role

As a Senior Data Engineer, you will design, build and maintain sophisticated data infrastructure supporting analytics, reporting and operational decision-making. You will work across the full data life cycle, developing robust data pipelines, implementing automation and supporting Real Time data streaming capabilities.

Responsibilities

• Design, develop and maintain scalable data pipelines using Python, Go and Azure Data Factory
• Develop ETL and ELT processes supporting structured, semi-structured and unstructured data
• Implement and manage hybrid data solutions across on-premises and Azure environments
• Build and maintain PostgreSQL database solutions
• Manage Elasticsearch deployments and optimisation
• Implement workflow automation using Azure Data Factory
• Develop and support Real Time event streaming solutions using Kafka and NATS
• Support security, authentication and access management using technologies such as Keycloak
• Collaborate with technical teams to deliver resilient, secure and scalable data platforms

Skills

• Strong commercial Python development experience
• Experience developing ETL/ELT solutions within enterprise environments
• Azure Data Factory expertise
• Experience with Kafka and/or NATS event streaming technologies
• PostgreSQL administration and development experience
• Elasticsearch experience
• Strong understanding of data security, access controls and platform governance
• Experience operating within hybrid cloud and on-premises environments
• Active SC Clearance

Desirable

• Azure certifications
• Experience with Go development
• DevOps and automation experience
• Experience supporting highly secure government or defence environments
• Willingness and eligibility to progress to DV Clearance

Benefits

• Up to £85,000 salary
• Work on nationally significant programmes
• Join a growing and highly respected consultancy
• Exposure to modern cloud, data and platform technologies
• Career development and certification support
• Opportunity to progress through higher levels of security clearance

Please note: Active SC Clearance is mandatory for this position. Candidates must also be willing to undergo DV clearance as part of future project requirements.