Senior Data Engineer

UK Home Office; Sheffield; Hybrid; GBP 50,000 - 57,000; Full time

Benefits offered by this job

Civil Service Pension

25 days annual leave

Hybrid working 60% in office

Job summary

UK Home Office is seeking a Senior Data Engineer to design and implement data flows that connect operational systems with analytics and BI platforms. You will lead data pipelines, collaborate with Data Science and Product Owners, and help shape service technology and data architecture.

You will mentor juniors, work in an agile team, and contribute to cost dashboards and cloud cost reporting for the organisation.

Qualifications

• Analytical thinking and problem solving for complex data flows.
• Experience with data modelling and ETL concepts.

Responsibilities

• Design and implement data streaming services and ETL processes.
• Build and maintain data pipelines, datasets and dashboards for cost dashboards and forecasts.
• Collaborate with finance, cloud platform teams and stakeholders to ensure data quality and traceability.
• Mentor junior team members and promote an agile way of working.

Job description

Location: Glasgow | Salford | Sheffield (hybrid with 60% office attendance)

Salary: £49,850 plus skills allowance of up to £7550 pending assessment

Advert Close Date: Thursday 24th September 11:55pm

Home Office Digital designs, builds and develops services for the rest of the department and for government. Every year our systems support up to 3 million visa applications, checks on 100 million border crossings, up to 8 million passport applications and deliver 140 million police checks on people, vehicles and property.

Senior Data Engineers design and implement complex data flows to connect operational systems with analytics and BI systems. You’ll build and manage data flows, collaborating with the Data Science community and Product Owners to identify strategies and services to best support business needs. You’ll help set the direction for service technology and data architecture, while mentoring junior team members, promoting collaboration and encouraging an agile approach to work.

You’ll get to work with some of the largest and most varied datasets around, and benefit from a wealth of continuous professional development resources and career opportunities. You’ll play a key role in delivering joined-up, intelligent services that unlock the value from data and deliver better outcomes for the UK.

This role sits in the Central FinOps team and plays a critical role in enabling accurate cloud cost allocation, recharge, and financial transparency across Home Office. The role focuses on collecting, transforming and maintaining large volumes of AWS and Azure cost and usage data, ensuring that cloud expenditure can be accurately attributed to portfolios, programmes and services. This supports the Central FinOps team's recharge and chargeback processes, providing a reliable basis for financial management and accountability across the organisation.

A key responsibility is the development and maintenance of data pipelines, datasets and reporting solutions that underpin cloud cost dashboards, forecasting, invoicing and recharge activities. The engineer works closely with finance teams, cloud platform teams and business stakeholders to ensure data quality, consistency and traceability. They support the production of recharge datasets and dashboards, enabling portfolio teams to understand their cloud consumption and associated costs.

The role also contributes to cost optimisation, tagging compliance, data governance and cloud unit economics. By automating data collection, improving reporting accuracy and developing analytical capabilities, the role helps Home Office manage significant public cloud expenditure effectively.

Senior Data Engineers design and lead the implementation of complex data flows to connect operational systems, data for analytics and BI systems.

As a Senior Data Engineer, you will identify, build, modify and manage data flows between complex systems using the appropriate solution.

You will be working with the Data Science community, who set the scope of data engineering work through user research and data pipelines. Through sharing work with Product Owners, you will identify strategies for how the service is best able to support business needs.

You will work with fellow members of the Data Engineering community to set the direction of the service technology and data architecture. You will also mentor more junior members of the team - promoting challenge, collaborating and encouraging an agile approach to working.

Main Responsibilities

• Analysing problems and experimenting with possible solutions to find the underlying causes of issues or discrepancies to identify problems and to assist in the development of innovative solutions.
• Analysing and reporting test activities and results, supporting data engineers and stakeholders on data analysis issues.
• Designing and implementing a data streaming service, including the development of new data models and ETL processes.
• Ensuring the successful delivery of completed data loads for customers, Data Analysts and Data Scientists, troubleshooting where required.
• Designing, building and testing data products and solutions that are complex or large scale, through full development, test and deployment lifecycles.
• Applying concepts and principles of data modelling and producing relevant and varied data models across multiple subject areas and providing guidance on how to use them.
• Working experience of finance processes, procedures, and controls (ideally in UK Government), processing financial transactions accurately and in line with delegated authorities, financial regulations, and audit requirements, including an ability to identify, investigate, and resolve financial discrepancies in a timely manner.
• Strong understanding of financial governance, risk management, and internal control frameworks, excellent attention to detail and commitment to accuracy in financial processing.
• Practical experience of receipting and invoicing, including purchase-to-pay (P2P) and order-to-cash (O2C) processes, working with finance systems and enterprise resource planning (ERP) platforms, e.g. Oracle.
• Experience of working collaboratively with suppliers, commercial teams, and operational stakeholders to ensure compliance with contractual and financial requirements, with an ability to communicate complex financial information clearly to a range of stakeholders.
• The delivery of projects in data analysis, solution design and end user reporting.
• Experience of any of the following: IT programming and automation, using tools such as industry standard ETL tools, network databases and scheduling and orchestration tools, using modern / open source programming languages in order to deliver data development / engineering products, using public cloud data technologies.
• Effectively managing and communicating with non-technical and senior stakeholders about performance and analysis.

Please note that this role requires Security Check (SC) clearance, which requires 5 years’ UK residency in the past 5 years.

Why work for us...

Find out more information at: Benefits - Home Office Careers, but some of the primary ones are:

• A Civil Service Pension with employer contribution rates of at least 28.97%.
• In-year reward scheme for one-off or sustained exceptional personal or team achievements.
• 25 days annual leave on appointment, rising with service.
• 8 days of public holidays, plus 1 additional privilege day.
• Where business needs allow, some roles may be suitable for a combination of office and home-based working. This is a non-contractual arrangement where all employees will be expected to spend a minimum of 60% of their working time in an office.