Senior Data Engineer: Scalable Pipelines & Data Quality

Government Recruitment Service; Leeds; On-site; GBP 60,000 - 80,000; Full time

Job summary

A governmental agency is seeking a skilled data engineer to build, maintain, and optimize scalable data pipelines and platforms in Leeds. Responsibilities include designing ETL pipelines, enforcing data quality standards, and collaborating with various data teams. Candidates should be adept at integrating data sources and automating processes. This role offers the chance to contribute to evidence-based policy through data management, with a focus on security and compliance.

Qualifications

• Experience in designing and operating ETL and streaming pipelines for large datasets.
• Ability to implement data modelling and quality standards.
• Familiarity with CI/CD tools and observability practices.

Responsibilities

• Design ETL and streaming pipelines for diverse datasets.
• Enforce data modelling and quality standards.
• Optimize performance on cloud and on-prem platforms.
• Integrate data sources using APIs and real-time methods.
• Automate deployment and monitoring processes.
• Collaborate to create reusable data products.
• Ensure security and compliance with standards.
• Mentor junior engineers and contribute best practices.

Skills

ETL and streaming pipeline design

Data quality standards enforcement

Cloud platform optimization

API integration

CI/CD automation

Collaboration with data teams

Data security compliance

Mentorship skills

Job description

A governmental agency is seeking a skilled data engineer to build, maintain, and optimize scalable data pipelines and platforms in Leeds. Responsibilities include designing ETL pipelines, enforcing data quality standards, and collaborating with various data teams. Candidates should be adept at integrating data sources and automating processes. This role offers the chance to contribute to evidence-based policy through data management, with a focus on security and compliance.