Senior Data Engineer: Real-Time Data & BI Lead

Allscreens Nationwide Ltd; Dundee; On-site; GBP 70,000 - 110,000; Full time

Job summary

Social Security Scotland is seeking a Senior Data Engineer to drive the design and delivery of robust, real-time data platforms that support analytics and BI for Scotland's digital services.

You will collaborate with Engineering, Cloud Services, Architecture and Applications to build scalable data flows, data products and conformed models that enable informed decision-making while promoting a positive, inclusive culture.

Qualifications

• Main duties include designing, developing and supporting real-time data streaming solutions using Apache Kafka to deliver scalable, reliable data integration.
• Design and implement scalable Python-based data processing, integration, and automation solutions.
• Lead technical discussions during sprint planning and agile ceremonies, collaborating with stakeholders.

Responsibilities

• Design, develop and support real-time data streaming solutions using Apache Kafka to deliver scalable, reliable data integration.
• Design and implement scalable Python-based solutions for data processing, integration and automation.
• Lead technical discussions during sprint planning, story refinement and other agile ceremonies, working closely with stakeholders.
• Analyse, profile and assess data from multiple complex sources to understand structure, quality and business value.
• Design, build and maintain complex, large-scale data products and integration services.

Job description

Social Security Scotland is seeking a Senior Data Engineer to drive the design and delivery of robust, real-time data platforms that support analytics and BI for Scotland's digital services.

You will collaborate with Engineering, Cloud Services, Architecture and Applications to build scalable data flows, data products and conformed models that enable informed decision-making while promoting a positive, inclusive culture.