Senior Data Engineer — Hybrid Cloud Pipelines

Methods; Gloucester; Hybrid; GBP 40,000 - 60,000; Full time

Benefits offered by this job

Development access to LinkedIn Learning

24/7 confidential employee assistance programme

Flexibility to work from home and part-time

25 days annual leave

Pension with employer contribution

Life Assurance

Private Medical Insurance

Job summary

Methods is looking for a Data Engineer to join our team in Gloucester. This permanent position offers the chance to work on client sites while managing data pipelines, storage solutions, and workflows mainly in hybrid environments.

The ideal candidate will have strong Python skills, experience with Azure Data Factory, and knowledge of event streaming technologies like Kafka. Benefits include flexible working options, 25 days of annual leave, and access to a range of wellness programs.

Qualifications

• Strong Python skills for data automation processes.
• Demonstrable experience in ETL or ELT workflows.
• Experience with Azure Data Factory is required.
• Proven ability with Kafka and NATS for event streaming.
• Knowledge of security practices with Keycloak.
• Background in managing Elasticsearch and PostgreSQL.

Responsibilities

• Design and maintain data pipelines using Python/Go/Azure.
• Implement hybrid cloud data storage solutions.
• Automate data flows with Azure Data Factory.
• Handle real-time data streams using event-driven technologies.
• Manage security setups and access controls.
• Develop and manage PostgreSQL databases.

Skills

Python

ETL/ELT

Azure Data Factory

Kafka

Data Security

PostgreSQL

Tools

Elasticsearch

Keycloak

Job description

Methods is looking for a Data Engineer to join our team in Gloucester. This permanent position offers the chance to work on client sites while managing data pipelines, storage solutions, and workflows mainly in hybrid environments.

The ideal candidate will have strong Python skills, experience with Azure Data Factory, and knowledge of event streaming technologies like Kafka. Benefits include flexible working options, 25 days of annual leave, and access to a range of wellness programs.