Data Engineer

The Green Recruitment Company; Southampton; On-site; GBP 42,000 - 45,000; Full time

Job summary

A leading environmental and sustainability business is seeking a Data Engineer to join their Data-Technology team. In this role, you will design, build, and maintain data solutions while ensuring data quality and supporting data-driven decision-making. Candidates should have 3-5 years of cloud-based data engineering experience and proficiency in tools like ADF and PySpark.

Benefits

Attractive company benefit package

Career development opportunities

Qualifications

• 3-5 years Cloud-based Data Engineering experience.
• Experience leading or working in an engineering team.
• Formal data engineering qualification desired.

Responsibilities

• Develop scalable data pipelines for diverse data integration.
• Maintain pipelines within Azure ADF and Synapse Analytics.
• Take ownership of data engineering projects and tasks.

Skills

Cloud-based data engineering tools

PySpark notebooks

Analytical skills

Problem-solving skills

Communication skills

Education

Bachelor’s degree in Computing or Software Development

Tools

ADF

Synapse Analytics

S3

Lambda

Azure DevOps

Job description

The Green Recruitment Company is working with an Environmental and Sustainability Business that supports and empower their customers journey to net zero. To join their Data-Technology team, we have an opportunity for a Data Engineer to help support the wider business (cross-functional) to meet their reporting and data requirements.

About the role:

• The Data Engineer will be working with the latest innovative technologies, to design, build and maintain data solutions, constructing process to surface data both internally for reporting purposes and externally through the customer portal.
• The Data Engineer will be responsible for developing scalable data pipelines to integrate diverse data sources whilst ensuring data quality under the framework of a new Data Platform for real time application integration and reporting, and will work closely with business stakeholders to support data-driven decision making by delivering clean, well-structured datasets that can be utlised for reporting purpose in a performent, secure way.

Key responsibilities:

• Taking full ownership of assigned projects and BAU tasks.
• Maintaining current pipelines within Azure ADF and Synapse Analytics.
• Build a process of transforming raw data from various CRM system into a harmonised and curated layer.
• Develop the usage of event driven topics for usage of various subscribers.
• Investigate and document Architectural Spikes to help foster best practice within the Data Team.
• Developing and creating data science tools to give a deeper understanding of the customer book.
• Recording and updating of work on Project Management System (Azure DevOps).
• Own and enhance the BAU runbook for engineering operations
• Develop the instrumentation and monitoring of IT automated tasks
• Taking a lead in the engineering function of the data team

Requirements:

Qualifications: Bachelor’s degree or above in Computing or Software Development or similar

Experience required:

• 3-5 years' Cloud-based Data Engineering experience,
• Previous experience of formal methodologies with data engineering
• Experience leading or working in an engineering team or function
• Previous experience of using the Azure Stack
• Experience working in a proactive analytics function
• Experience of working in the Utilities sector
• Experience leading technical projects

Skills & Technologies required:

• Proficiency in cloud-based data engineering tools (ADF, Synapse Analytics, S3, Lamda)
• Proficiency in using PySpark notebooks for ELT.
• Fostering and cultivating a culture of best practices
• Strong analytical and problem-solving skills.
• Ability to work independently and as part of a functional and cross-functional team
• Excellent communication and documentation skills.
• Proven ability to design, evaluate and score engineering options
• Formal data engineering qualification

On offer: Salary £42 000 - £45 000 (depending on experience) with an attractive company benefit package & career development