Senior Data Engineer — Hybrid Cloud Pipelines

Methods; Malvern; Hybrid; GBP 50,000 - 75,000; Full time

Benefits offered by this job

Access to LinkedIn Learning

24/7 confidential employee assistance programme

Flexible working including home working

25 days annual leave plus bank holidays

Pension with employer contribution

Private medical insurance

Job summary

Methods is seeking a skilled Data Engineer to join their team in Malvern on a permanent basis. This role includes some client-site work and regular visits to the main offices in London. The ideal candidate will engage in developing data pipelines, managing security, and designing PostgreSQL databases, leveraging tools such as Azure Data Factory and event-driven technologies.

You will benefit from a collaborative and supportive environment with various employee benefits, including professional development opportunities, flexible working arrangements, and health insurance.

Qualifications

• Strong Python skills for data processing automation.
• Demonstrable experience in developing and optimizing ETL or ELT workflows.
• Practical experience with Azure Data Factory is required.
• Proven ability in managing event streaming platforms.
• Knowledge of implementing security practices, including Keycloak.
• Strong background in managing Elasticsearch and PostgreSQL.

Responsibilities

• Design, construct, and maintain data pipelines using Python/Go/Azure Data Factory.
• Implement and manage data storage solutions across platforms.
• Employ tools to automate data flows and manage workflows.
• Utilize event-driven technologies to handle real-time data streams.
• Manage security setups and access controls for data integrity.
• Design and develop PostgreSQL databases for diverse deployments.

Skills

Python

ETL/ELT

Azure Data Factory

Kafka

NATS

PostgreSQL

Elasticsearch

Data Security

Job description

Methods is seeking a skilled Data Engineer to join their team in Malvern on a permanent basis. This role includes some client-site work and regular visits to the main offices in London. The ideal candidate will engage in developing data pipelines, managing security, and designing PostgreSQL databases, leveraging tools such as Azure Data Factory and event-driven technologies.

You will benefit from a collaborative and supportive environment with various employee benefits, including professional development opportunities, flexible working arrangements, and health insurance.