Data Engineer

Women in Data®; Greater London; Hybrid; GBP 55,000 - 67,000; Full time

Benefits offered by this job

Civil Service Pension

25 days leave

Hybrid working

Flexible working arrangements

Job summary

Department for Education is hiring a Data Engineer to join the Data Directorate and deliver end-to-end data solutions. You will work in a multidisciplinary team on data pipelines, translate requirements into robust engineering solutions, and contribute to the department's data strategy with modern cloud technologies in a hybrid working pattern.

The role requires experience with SQL, Python and Spark, cloud platforms (Azure/AWS/Google Cloud) and DevOps practices.

Qualifications

• Proficiency in SQL, Python and Spark (Pyspark/Spark SQL).
• Experience designing robust data engineering solutions on cloud platforms such as Azure, AWS or Google Cloud.
• Experience with Agile/DevOps practices including CI/CD and automation.

Responsibilities

• Work in a multidisciplinary team to develop and deliver end-to-end data solutions.
• Translate user requirements and data models into robust data engineering solutions.
• Deliver data engineering work aligned with project goals and delivery cycles.

Skills

SQL

Python

Apache Spark

Tools

Azure

Databricks

Azure Data Factory

Job description

• Location: Coventry, Manchester, Sheffield, Darlington, London
• Salary: up to £61,340

Job summary

IMPORTANT - This vacancy is using the Civil Service Numerical and Verbal tests. The deadline for completing the tests is the same as the vacancy closing date. Please ensure you complete the tests with sufficient time to complete the written application.

At DfE, we are proud of the commitment we make to diversity and inclusion and of the progress we have made. We have active & vibrant staff networks, special leave policies and workplace adjustments put in place for those who need them. We are continuing to build a diverse DfE, in an inclusive environment which nurtures and realises potential in all, at all levels.

At our heart, we are the department for Opportunity. We enable children and learners to thrive, by protecting the vulnerable and ensuring the delivery of excellent standards of education, training and care.

Data is integral to achieve this vision and the roles on offer are part of the Data Directorate within DfE. As a directorate, we provide a full range of data sourcing, management, governance, sharing, statistics, and visualisation capabilities and products to users across DfE, the education sector, cross-government, and citizens. We’re seeking highly capable Data Engineers to help us achieve our vision.

We’re striving to create a place that’s great to work in. We offer forward-thinking ways of working and training opportunities, as well as providing support from specialist colleagues. On top of that, you’ll find flexible working, an inclusive culture and a place where your skills can bring tangible change.

As a Data Engineer, you will be joining a team of talented data engineering professionals, who are committed to delivering high-impact, secure data solutions to achieve departmental and directorate objectives. You will:

• Work as part of a multidisciplinary team in a large and complex data environment to develop and deliver end-to-end data solutions.
• Translate user requirements and data models into effective, robust data engineering solutions in line with build standards and ensure code is production ready.
• Deliver data engineering solutions in line with project goals and delivery cycles, while working openly and collaboratively in our team.
• Be proactive and want to make an impact, and this may sometimes involve challenging current methods.
• Help build and maintain our data engineering community to facilitate knowledge sharing, innovation, and collaboration across the department.
• Work with new technologies such as Azure cloud-based tech stack, Databricks, Azure Data Factory, Agile ways of working, and practices such as CI/CD and DevOps. We use a combination of tooling, data quality processes, scripting, and modern cloud technologies to drive the engineering solutions.

Person specification

We are looking for experience in:

• Designing and building robust and re-usable data engineering solutions on any of the following cloud platforms – Azure, AWS, Google Cloud.
• Working with common data types and industry standard architectural patterns such as data warehouses/lake houses etc. - to service operational and/or analytical business needs.
• Proficiency in one or more of the following – SQL, Python, Apache Spark (Pyspark, Spark SQL)
• Performing troubleshooting, root-cause analysis and improving data quality in the data products.
• Delivering through Agile/DevOps working practices in multi-disciplinary teams – CI/CD, Scrum, Automation.
• Staying up-to-date with latest tools and technologies and building proof-of-concepts using those technologies to solve a business problem.

We are keen to hear about the following desirable skills, although these are not essential:

• Experience with Databricks, Informatica DEI and Azure Data Factory.
• Experience of implementing dimensional models.

Desirable criteria will only be assessed in event of a tie break situation to make an informed decision.

Behaviours

We'll assess you against these behaviours during the selection process:

• Making Effective Decisions

Alongside your salary of £44,305, Department for Education contributes £12,835 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides (opens in a new window).

Applicants currently holding a permanent post in the Civil Service should note that, if successful, their salary on appointment would be determined by the Department’s transfer / promotion policies.

As a member of the DfE, you will be entitled to join the highly competitive Civil Service Pension Scheme, which many experts agree is one of the most generous in the UK.

You will have 25 days leave, increasing by 1 day every year to a maximum of 30 days after five years’ service. In addition, all staff receive the King’s Birthday privilege holiday and 8 days’ bank and public holidays.

We offer flexible working arrangements, such as job sharing, term-time working, flexi-time and compressed hours.

Most DfE employees will be working a hybrid pattern, spending at least 60% of their time in an office or work setting. Changes to these working arrangements are available in exceptional circumstances but must be agreed with the line manager and in line with the requirements of the role.

Travel to your primary office location will not be paid for by DfE, but costs for travel to an office which is not your main location will be covered.

As an organisation, which exists to support education and lifelong learning, we offer our staff excellent professional development opportunities.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).