SC Cleared Python Data Engineer — Azure & PySpark

Brightbox GRP; Greater London; On-site; GBP 80,000 - 100,000; Full time

Job summary

A leading technology solutions firm in Greater London is seeking a SC Cleared Python Data Engineer to develop and maintain complex data flows in Azure. Candidates should excel in Python and have experience with PySpark, focused on building scalable data pipelines and optimizing Delta Lake storage. The ideal applicant will collaborate closely with engineering and DevOps teams to ensure quality and performance in their deliverables. This role offers a contract of £400 to £458 per day (Inside IR35).

Qualifications

• SC Clearance is essential.
• Experience in data engineering roles with a focus on Python and data pipelines.
• Competency in writing unit and BDD tests.

Responsibilities

• Design and build robust data pipelines using Python and PySpark.
• Develop complex data flows focusing on clean, testable code.
• Optimize Delta Lake tables for performance.

Skills

Strong Python skills

Experience with PySpark

Knowledge of Docker

CI/CD pipeline collaboration

Tools

Azure

Delta Lake

Behave for testing

Job description

A leading technology solutions firm in Greater London is seeking a SC Cleared Python Data Engineer to develop and maintain complex data flows in Azure. Candidates should excel in Python and have experience with PySpark, focused on building scalable data pipelines and optimizing Delta Lake storage. The ideal applicant will collaborate closely with engineering and DevOps teams to ensure quality and performance in their deliverables. This role offers a contract of £400 to £458 per day (Inside IR35).