Senior Data Engineer: Scalable Pipelines & Data Quality

Government Recruitment Service; Swansea; On-site; GBP 40,000 - 60,000; Full time

Job summary

A government agency in the United Kingdom is looking for a skilled data engineer to design and build scalable ETL and streaming pipelines. The role involves ensuring data quality, compliance with standards, and collaborating with cross-functional teams to create reusable data products. Candidates should have experience in optimizing data processing and integrating various data sources. This position is ideal for someone with strong technical abilities and a passion for data-driven decision-making.

Qualifications

• Experience in designing and building ETL and streaming pipelines.
• Strong understanding of data quality and compliance standards.
• Proficient in automation tools for deployment and monitoring.

Responsibilities

• Design, build and operate ETL and streaming pipelines for large datasets.
• Implement data modelling and quality standards.
• Optimise performance on cloud and on-prem platforms.
• Integrate multiple data sources through various methods.
• Automate testing and monitoring processes.
• Collaborate with various teams to deliver data products.
• Ensure compliance with government data standards.
• Mentor junior engineers.

Skills

ETL development

Data modelling

Cloud computing

API integration

CI/CD

Data security compliance

Job description

A government agency in the United Kingdom is looking for a skilled data engineer to design and build scalable ETL and streaming pipelines. The role involves ensuring data quality, compliance with standards, and collaborating with cross-functional teams to create reusable data products. Candidates should have experience in optimizing data processing and integrating various data sources. This position is ideal for someone with strong technical abilities and a passion for data-driven decision-making.