Senior Data Engineer

Government Recruitment Service; Newcastle upon Tyne; On-site; GBP 55,000 - 75,000; Full time

Job summary

A public sector organization is seeking a Senior Data Engineer to lead efforts in developing cloud-based solutions that are critical for business operations. This role involves mentoring a team of engineers and collaborating with cross-functional teams to ensure data standards and quality. You will be responsible for technical leadership, transforming user requirements into robust ETL solutions, and improving data journeys for efficiency and effectiveness. Opportunities for professional development and the use of cutting-edge tools are available.

Benefits

Access to learning and development resources

Attendance at industry conferences

Flexible working conditions

Qualifications

• Proven experience in data engineering and building ETL pipelines.
• Ability to lead teams and mentor junior engineers.
• Strong understanding of data tools, sources, and best practices.

Responsibilities

• Lead a team of Data Engineers in developing and maintaining ETL pipelines.
• Ensure adherence to data engineering standards and best practices.
• Translate user requirements into effective data solutions.

Skills

Cloud-based solutions

Data Engineering

ETL solutions

Artificial intelligence

Machine learning

Robotic process automation

Log analytics

Team leadership

Education

Relevant degree in Computer Science or related field

Tools

Elastic Stack

Qlik Sense

Jira

SharePoint

Teams

Job description

Are you a Data Engineer interested in working within a team on the latest cutting-edge technologies building business critical cloud-based solutions?

Are you interested in being part of a Live Service team, responsible for the operational stability, ongoing development and maintenance of cutting edge, innovative cloud-based solutions?

If so, The Garage has an exciting opportunity for you! Both of the above are seen as business-critical teams delivering maximum value across DWP.

The Garage is an established delivery unit, providing DWP with a range of digital transformation and automation capabilities including but not limited to Robotics Process Automation, Computer Vision, Object Character Recognition (OCR), Natural Language Processing, Machine Learning (ML), Generative AI and Deep Learning for Digital and Operational colleagues to support, enable, enhance and/or accelerate digital and service transformation.

To date, The Garage has developed and deployed over 80 solutions that have processed over 74 million transactions. Recognised as a government leader in intelligent automation, The Garage is also acknowledged as leading the way across industry.

Please note this role requires you to pass Security Check clearance. For further information, please see 'Selection process details'.

As a Senior Data Engineer, you are responsible for leading, managing and delivering all aspects of data development activities to meet customer needs to agreed timescales.

You will own the overall Data Engineering Development process within Data Engineering along with setting the strategies for products and the wider community.

As a Senior Data Engineer, you will provide technical leadership to a team of Data Engineers along with providing oversight to the products they develop.

You’ll lead a small team of Data Engineers and collaborate closely with Software Engineers, DevOps Engineers, and Service Management teams to build and maintain high-quality ETL pipelines and data strategies.

Your work will directly support the delivery of cutting-edge solutions using technologies such as artificial intelligence, machine learning, and robotic process automation. You will have line management responsibilities, mentoring and coaching others, helping to build a strong Data community.

To help you really make a difference, you’ll be using the best tools and equipment for the job. We use Elastic Stack to conduct log analytics as a search and real-time visualisation tool. We use Qlik Sense as our primary BI tool for data integration and dashboard creation. We plan and track work in Jira, we document in SharePoint, we communicate using Teams. We do it all on MacBook Pro devices.

You will also have access to fantastic learning and development resources; conferences with industry expert speakers, comprehensive training, and subscriptions to digital training solutions. Your roles and responsibilities include:

• Lead the redevelopment of existing data journeys, providing leadership to improve, performance, resilience, or to account for system, tooling or platform change on code frameworks and practices to meet all user needs.
• Develop, own, maintain and ensure adherence to standards across the Data Engineering community in proactively preventing issues and problems with data, outputs, tools and pipelines.
• Provide technical leadership, setting engineering team and community direction to ensure the products that are developed conform to data standards and strategic principles
• Responsible for translating user requirements and data designs into effective, robust data ETL solutions in line with build standards and ensure code is production ready and properly assured.
• Accountability and ownership for BI reporting strategy and ensure continuous improvement, maintenance and deletion/archiving activities are undertaken. Works with other disciplines to ensure best practice is embedded.
• Design and undertake complex analysis to resolve data and pipeline problems, issues; ensuring quality outputs can be supplied to researchers and other customers.
• Communicating with stakeholders at all levels: managing expectations around delivery of data, products and data pipelines; facilitating discussions across high risk and complex projects or under constrained delivery timescales.
• Design and undertake complex analysis and build capability to resolve data and pipeline problems, issues; ensuring quality outputs can be supplied to researchers and other customers.
• Lead the redevelopment of existing data journeys, providing leadership to improve, performance, resilience, or to account for system, tooling or platform change on code frameworks and practices to meet all user needs.
• Develop, own, maintain and ensure adherence to standards across the Data Engineering community in proactively preventing issues and problems with data, outputs, tools and pipelines.
• Provide technical leadership, setting engineering team and community direction to ensure the products that are developed conform to data standards and strategic principals.
• Responsible for translating user requirements and data designs into effective, robust data ETL solutions in line with build standards and ensure code is production ready and properly assured.
• Accountability and ownership for BI reporting strategy and ensure continuous improvement, maintenance and deletion/archiving activities are undertaken. Works with other disciplines to ensure best practice is embedded.
• Design and undertake complex analysis to resolve data and pipeline problems, issues; ensuring quality outputs can be supplied to researchers and other customers.
• Communicating with stakeholders at all levels: managing expectations around delivery of data, products and data pipelines; facilitating discussions across high risk and complex projects or under constrained delivery timescales.
• Design and undertake complex analysis and build capability to resolve data and pipeline problems, issues; ensuring quality outputs can be supplied to researchers and other customers.
• Demonstrating a deep understanding of relevant data sources, tools and systems, using appropriate approaches for verifying and validating data and influencing senior stakeholders in data approaches. Coaching and mentoring other junior Data Engineers.

For further information, please see the Government Digital and Data Profession Capability Framework which is the official standard used by the UK government to define roles, skills and career paths for professionals working in digital, data and technology: Government Digital and Data Profession Capability Framework - Government Digital and Data Profession Capability Framework