Data Engineer - SC Clearance

Stott and May; Leeds; On-site; GBP 45,000 - 75,000; Full time

Job summary

An established industry player is seeking a talented Data Engineer to join their innovative team. This role involves transforming proof-of-concept projects into robust production environments using cutting-edge Microsoft Azure technologies. You will develop scalable data pipelines, enhance data security, and implement efficient data processing techniques. With a focus on data-driven transformation for the public sector, this position offers the chance to work at the forefront of technology while enjoying a range of flexible benefits, including employee share plans and private healthcare. If you are passionate about data and technology, this opportunity is perfect for you.

Benefits

Employee share plans

Flexible benefits including green car scheme

Private healthcare

Lifestyle discounts

Annual performance-related bonus

Qualifications

• Experience with Azure Data Factory and data pipeline development.
• Strong knowledge of data security and masking techniques.

Responsibilities

• Transforming proof-of-concept into a robust production environment.
• Developing scalable Azure Data Factory pipelines for data operations.

Skills

Azure Data Factory

Data Security

Data Ingestion Processes

Performance Tuning

Data Masking Techniques

SQL

Microsoft Entra ID

Job description

Join a leading digital intelligence organisation working at the forefront of data-driven transformation for the public sector. We are looking for a Data Engineer with active SC Clearance to help evolve a successful proof-of-concept into a fully productionised and supported data platform using cutting-edge Microsoft Azure technologies.

What You’ll Be Doing:

1. Transforming proof-of-concept into a robust, scalable, and secure production environment.
2. Developing scalable Azure Data Factory pipelines.
3. Enhancing security, manageability, and cost-efficiency of data operations.
4. Building reusable, templated data ingestion processes.
5. Applying data segregation, masking, and pseudonymisation techniques.
6. Implementing incremental dimension creation to reduce compute and data processing costs.
7. Using managed identities or service principals for secure authentication.
8. Optimizing parallelism and performance tuning for ADF pipelines.
9. Creating templated, reusable ingestion patterns.
10. Integrating with Microsoft Entra ID (formerly Azure AD).
11. Implementing STAR schema for analytic models.
12. Standardising naming conventions and schema organisation.
13. Migrating data processing logic from Power BI to SQL layer.
14. Applying robust data security, masking, and pseudonymisation techniques.

What’s In It For You:

• Employee share plans
• Flexible benefits including green car scheme, private healthcare, and lifestyle discounts
• Annual performance-related bonus (where eligible)

Additional Details:

• Seniority level: Mid-Senior level
• Employment type: Full-time
• Job function: Information Technology
• Industries: Technology, Information and Media, IT Services and IT Consulting, Defense and Space Manufacturing