Data engineer in local government

Barnet Council; Greater London; On-site; GBP 65,000 - 90,000; Full time

Benefits offered by this job

LGPS

Гибкий график работы

Обучение и развитие

Программы благополучия сотрудников

Скидки у партнеров

Job summary

Barnet Council приглашает опытного Data Engineer присоединиться к Hub Insight & Intelligence для разработки и поддержки современных дата-решений. Вы будете строить и оптимизировать конвейеры и платформы данных, обеспечивая качество и безопасность данных для аналитики, ИИ и ML.

Ключевые задачи включают поддержку Azure Data Platform, переход к Fabric и тесное сотрудничество с внутренними командами и внешними поставщиками. Требуется глубокий опыт в Python, SQL и облачных сервисах.

Qualifications

• Высшее образование или эквивалентный опыт в области данных.
• Опыт работы с облачными платформами и современными инструментами обработки данных (Azure, Data Factory, Synapse, Spark, Snowflake, Databricks).
• Опыт проектирования и поддержки безопасных конвейеров ETL и lakehouse-архитектур.
• Умение писать на Python и SQL, работа с реляционными и NoSQL БД.
• Опыт работы с GDPR/информационной безопасностью и требованиями.
• Способность объяснять сложные технические вопросы коллегам без технического бэкграунда.

Responsibilities

• Поддержка и развитие Azure Data Platform и переход к Microsoft Fabric.
• Проектирование надёжных ETL-процессов и повторно используемых рабочих потоков данных.
• Обеспечение качества, доступности и безопасности данных.
• Управление данными проектами и взаимодействие с поставщиками/заинтересованными сторонами.
• Разработка и распространение стандартов и практик Data Engineering в Barnet.

Skills

Azure Data Platform

Python

SQL

ETL

Data Pipelines

NoSQL

Databricks

Spark

Cloud Data Engineering

Education

Уровень бакалавра или эквивалент

Tools

Microsoft Azure

Data Factory

Synapse

Snowflake

Databricks

Power BI

Job description

Описание

This is an opportunity to play a leading role in developing Barnet’s modern data capability and helping the council gain greater value from its information. As a Data Engineer in our Insight & Intelligence Hub, you will build, connect and improve the data pipelines, platforms and products that support better decisions, more efficient services and improved outcomes for residents.

Задачи

You will help maintain our Microsoft Azure Data Platform and support its transition to Microsoft Fabric, integrating data from operational, supplier-hosted and cloud-based systems. You will design robust ETL processes, APIs and reusable data workflows; improve data quality, reliability and accessibility; and prepare trusted datasets for analytics, AI and machine learning. Working across the council, you will translate user needs into secure and scalable technical solutions while ensuring compliance with UK GDPR, information governance and security requirements. You will also lead complex data engineering projects, manage relationships with suppliers and stakeholders, and help establish strong standards and practices for data engineering across Barnet.

Требования

You will be an experienced Data Engineer with strong applied knowledge of modern data engineering technologies, cloud platforms and production data services. You will be educated to degree level, have an equivalent qualification, or demonstrate relevant professional experience. Strong experience of cloud data engineering platforms and tools, such as Microsoft Azure, Data Factory, Synapse, Spark, Snowflake or Databricks. Experience of designing and maintaining secure data pipelines, ETL processes and lakehouse architectures. Proficiency in Python and SQL, together with experience of relational and NoSQL databases. Experience of integrating, cleaning, modelling and scaling complex datasets from operational systems. Knowledge of version control, automated testing, metadata management and production support. Experience of leading data projects and delivering reliable data products or services. A sound understanding of UK GDPR, privacy by design, information security and the responsible handling of sensitive data. The ability to explain complex technical issues clearly to non-technical colleagues and influence stakeholders at different levels. Strong problem-solving, prioritisation and relationship-building skills, with a commitment to continuous improvement and collaborative working. You will bring curiosity, personal drive and an outcomes-focused approach, alongside a commitment to inclusion, professional development and sharing knowledge with others. Будет плюсом: Experience of Microsoft Fabric, event streaming, machine-learning datasets, Power BI or other visualisation tools would be an advantage

Условия

31 days annual leave, plus public and bank holidays Access to the Local Government Pension Scheme , which provides a valuable guaranteed income in your retirement together with security for your dependents A comprehensive Flexible Working Policy which supports work-life balance by empowering staff and management to make decisions around when, where, and how work is done A vast range of lifestyle discounts from major retailers, supermarkets, energy suppliers and more Broad range of payroll benefits including cycle to work, eye care vouchers, travel and gym membership Excellent training and development opportunities Employee well- being training programs including confidential employee assistance