Senior Data Engineer — Hybrid Cloud Pipelines

Methods; Worcester; Hybrid; GBP 45,000 - 65,000; Full time

Benefits offered by this job

24/7 employee assistance programme

Flexible working options

Development access to LinkedIn Learning

Social activities like office parties

25 days annual leave plus bank holidays

2 paid volunteering days annually

Pension scheme with employer contributions

Private medical insurance

Job summary

Methods in Worcester is seeking a Data Engineer to design and manage data pipelines and workflows. The ideal candidate should possess strong Python skills, experience with Azure Data Factory, and a solid understanding of ETL/ELT processes.

Joining Methods offers a collaborative environment with opportunities for project involvement and personal development. The role includes flexibility in working arrangements and a comprehensive benefits package, including 25 days vacation and private medical insurance.

Qualifications

• Strong Python skills for scripting and automating data processes.
• Experience with ETL or ELT workflows in hybrid cloud environments.
• Proficiency in using Azure Data Factory for workflow automation.
• Ability to manage event streaming platforms like Kafka and NATS.
• Knowledge of security practices and tools like Keycloak.
• Experience with managing Elasticsearch and PostgreSQL.

Responsibilities

• Develop and manage data pipelines using Python/Go/Azure Data Factory.
• Implement and manage Azure data storage solutions.
• Automate data flows and manage workflows using Azure Data Factory.
• Utilize event-driven technologies for real-time data streams.
• Manage security setups and access controls for data integrity.
• Design and develop PostgreSQL databases for high performance.

Skills

Python

ETL/ELT

Azure Data Factory

Event Streaming

Data Security Practices

PostgreSQL

Elasticsearch

Job description

Methods in Worcester is seeking a Data Engineer to design and manage data pipelines and workflows. The ideal candidate should possess strong Python skills, experience with Azure Data Factory, and a solid understanding of ETL/ELT processes.

Joining Methods offers a collaborative environment with opportunities for project involvement and personal development. The role includes flexibility in working arrangements and a comprehensive benefits package, including 25 days vacation and private medical insurance.