Senior Data Engineer

Great British Energy Group; Greater London; Remote; GBP 70,000 - 110,000; Full time

Job summary

The Ministry of Justice is seeking a Senior Data Engineer for a six-month contract, primarily remote with occasional visits to London. You will design, build and maintain automated data pipelines that ingest data from SplashBI into the programme's AWS data platform, enabling migration readiness and analytics across MoJ services.

You will work with Data Analysts and stakeholders, implement IaC for AWS, and apply CI/CD best practices.

Qualifications

• Strong hands-on AWS Data Engineering experience building robust, scalable pipelines.
• Experience with Python and SQL in a data engineering role.
• Ability to work with diverse data sources and support migration readiness.

Responsibilities

• Design, build and maintain automated data pipelines ingesting SplashBI data into AWS.
• Collaborate with Data Analysts to translate needs into data solutions.
• Maintain secure, scalable cloud infrastructure using IaC and best practices.
• Develop and optimise code, tools and services for analysts to access data.
• Monitor and optimise pipelines to meet programme timelines.

Skills

AWS

Python

SQL

Data Pipelines

ETL

Tools

SplashBI

GitHub Actions

Job description

On behalf of The Ministry of Justice, we are looking for a Senior Data Engineer (Inside IR35) for a 6 month contract based primarily remote (occasional travel to MoJ offices and team meetings, typically in London).The Synergy Programme is a large-scale transformation programme consisting of four Government departments, MOJ (Ministry of Justice), DWP (Department for Work and Pensions), DEFRA (Department for Environment, Food and Rural Affairs) and HO (Home Office). They will come together to replace their current back-office systems and introduce new ways of working for 220,000 civil servants combined. High-quality, accurate data is fundamental to ensuring a successful transition and enabling effective business processes, reporting and operations within the new ERP platform.The Data Engineer will play a critical role in supporting the MoJ contribution to the Synergy Programme by providing the technical capability required to extract, transform, manage and prepare source data for analysis. This role directly supports key programme priorities, including data quality improvement and data migration readiness.This role requires strong hands-on AWS Data Engineering experience, with a particular focus on building and maintaining automated data pipelines and cloud-based data solutions using AWS services, Python and SQL.As a Senior Data Engineer, your main responsibilities will include:* Designing, building and maintaining robust, scalable and automated data pipelines that extract source data from SplashBI and load it into the programme's AWS data platform.* Working closely with Data Analysts to understand user needs and data requirements, translating them into technical requirements and solutions.* Identifying opportunities to improve, automate, reuse and re-engineer data engineering processes to increase efficiency, enable scaling and reduce manual effort.* Maintaining secure and sustainable AWS cloud infrastructure using infrastructure-as-code principles and best practice development approaches, deploying solutions using a range of AWS services in a reproducible and sustainable manner.* Developing and optimising code, tools and services that enable analysts to access and use data effectively and efficiently.* Proactively monitor, maintain and optimise data pipelines to ensure source data is available to analysts in line with programme timelines and migration readiness milestones.* Operate effectively in a fast-paced programme environment, managing competing priorities and delivering technical solutions to demanding timescales.Essential:* Strong hands on AWS Data Engineering experience, including developing, maintaining and optimising robust and scalable automated data pipelines using Python, SQL and AWS cloud services.* Experience working in a technical team, preferably one that uses Python and SQL, as well as strong technical skills in these languages.* Working with a range of data sources and source systems and integrating data for analysis and migration purposes including data wrangling and analysis activities.* Understanding of the software development lifecycle and applying software development best practices, including version control (e.g. git), automated testing (e.g. pytest) and CI/CD (e.g. GitHub Actions).* Experience using infrastructure-as-code approaches to deploy and manage AWS cloud infrastructure.* Supporting data quality, migration or business intelligence activities through effective data engineering practices.* Working with technical and non-technical stakeholders to understand requirements and deliver effective data solutions.* Experience working in a fast-paced delivery environment, managing multiple priorities and delivering high-quality outputs to challenging deadlines.* Proven ability to proactively manage and maintain data pipelines, ensuring timely and reliable access to data for analytical and operational needs.* Ability to identify, investigate and resolve data pipeline issues with minimal disruption to programme delivery.PLEASE NOTE: SC Clearance is an essential requirement for this role, as a minimum you must be willing & eligible to undergo checks. Please note, due to the exceptional requirements of this position (short-term nature of this role and speed at which we require a postholder in situ) preference may be given to candidates who meet all of the essential criteria and hold active security clearance.Desirable:* Oracle E-Business Suite (EBS) experience across HR, Finance and Procurement.* SplashBI experience.* ERP transformation experience.* Data migration experience.Please be aware that this role can only be worked within the UK and not Overseas.Disability ConfidentAs a member of the Disability Confident Scheme, MOJ guarantees to interview all candidates who have a disability and who meet all the essential criteria for the vacancy. In cases where we have a high volume of candidates who have a disability who meet all the essential criteria, we will interview the best candidates from within that group. This scheme encourages candidates with a disability and/or neurodivergence to apply. In exceptional circumstances, we may also need to apply the desirable criteria in our shortlisting process which may include holding active security clearance.Armed Forces CovenantThe Ministry of Justice guarantees to interviewveterans or spouses / partners of military personnel who meet all the essential criteria for the vacancy. In cases where we have a high volume of ex-military candidates / military spouses or partners, who meet all of the essential criteria, wewill interview the best candidates from within that group. In exceptional circumstances, we may also need to apply the desirable criteria in our shortlisting process which may include holding active security clearance.In applying for this role, you acknowledge the following \"this role falls in scope of the Off Payroll Working in the Public Sector legislation. Any rates of payment quoted will reflect the gross rate per day for the assignment and will be subject to appropriate taxes and statutory costs. As such the payment to the intermediary and your income resulting from this contract will be different\".