Jobs in Warrington, United Kingdom
SC Cleared Python Data Engineer - Azure
Montash
Stone Wool Specification & Development Manager
Saint Gobain
Senior Contracts & Operations Lead – East of England Remote
Advance
Specification and Development Manager Stone Wool (South West)
Saint Gobain
Global RME Vendor Manager - Reliability & Operations Leader
Amazon
Discover more opportunities than anywhere else. Find more jobs nowPortfolio Mgr, RME Vendor Mgmt, Global Reliability Maintenance Engineering (RME)
Amazon
Remote Business Development Manager – Electrical Solutions
Bender UK
Apprentice Materials & Building Products Operative
Oxfordshire City Council
Connect with headhunters to apply for similar jobsBusiness Development Manager
Bender UK
Apprentice Operative Materials / Building Products - tarmac/TP/67621/7321
Oxfordshire City Council
Telecoms Transformation Programme Lead - Cloud & Data
Savant Recruitment
Remote Cloud Software Engineer: Scale, Secure & Modernize
Savant Recruitment
Cloud Software Engineer
Savant Recruitment
Programme Delivery Manager
Savant Recruitment
Clinical Trial Manager, Infectious Disease & Immunology
Medpace
Clinical Trial Manager - UK (London) - Infectious Disease / Immunology
Medpace
Strategic Partnerships & Operations Lead — Flexible Work
Advance
Remote UK Resident Director Nominee — Flexible, Low-Commitment
DevFinders
Aviation Quality Manager — Airport Project (Part-Time Telework)
Bechtel Oil, Gas & Chemicals Incorporated
Quality Manager, Aviation
Bechtel Oil, Gas & Chemicals Incorporated
65m Private Yacht Chef - Maldives Winter Charter
Leaman Consulting LLP
Temporary Chef Needed – 65m Private Yacht | Maldives
Leaman Consulting LLP
Python Data Engineer - Azure
Job summary
A leading consulting agency is looking for an SC Cleared Python Data Engineer to design and deliver scalable data pipelines using PySpark and Azure services. The ideal candidate will have strong Python expertise and experience in a cloud-native environment. Responsibilities include developing data ingestion pipelines, optimising Delta Lake transactions, and collaborating with multi-disciplinary teams. This remote position offers a challenging and dynamic work environment.
Qualifications
- Strong Python skills for production-quality code.
- Experience with PySpark and performance tuning.
- Familiarity with Behave for testing frameworks.
- Understanding of Delta Lake and its transactions.
- Experience with Docker in development workflows.
- Hands-on experience with Azure services.
Responsibilities
- Design, develop, and maintain data ingestion pipelines.
- Articulate PySpark architecture and performance to stakeholders.
- Implement unit and BDD testing with effective mocking.
- Design and optimise Delta Lake tables for performance.
- Build Docker environments for deployment.
- Integrate Azure services and support CI/CD pipelines.
Skills
Job Title: SC Cleared Python Data Engineer - Azure
Contract: 12 months
Location: UK Based - Remote
Start: January 2026
Clearance: Active SC Clearance required
We are seeking an SC Cleared Python Data Engineer with strong hands‑on experience in PySpark, Delta Lake, and Azure cloud services. The role focuses on designing and delivering scalable, well‑tested data pipelines, with particular importance placed on the ability to understand, explain, and design PySpark architectures and demonstrate deep, production‑grade Python expertise.
You will work in a containerised, cloud‑native environment, delivering maintainable, configurable, and test‑driven solutions as part of a multi‑disciplinary engineering team.
- Design, develop, and maintain data ingestion and transformation pipelines using Python and PySpark.
- Clearly articulate PySpark architecture, execution models, and performance considerations to both technical and non‑technical stakeholders.
- Implement unit and BDD testing (Behave or similar), including effective mocking and dependency management.
- Design and optimise Delta Lake tables to support ACID transactions, schema evolution, and incremental processing.
- Build and manage Docker‑based environments for development, testing, and deployment.
- Develop configuration‑driven, reusable codebases suitable for multiple environments.
- Integrate Azure services including Azure Functions, Key Vault, and Blob/Data Lake Storage.
- Optimise Spark jobs for performance, scalability, and reliability in production.
- Collaborate with Cloud, DevOps, and Data teams to support CI/CD pipelines and environment consistency.
- Produce clear technical documentation and follow cloud security and data governance best practices.
- Strong Python expertise, with a demonstrable depth of experience in designing modular, testable, production‑quality code.
- Proven experience explaining and designing PySpark architectures, including distributed processing and performance tuning.
- Hands‑on experience with Behave or similar BDD frameworks, including mocking and patching techniques.
- Solid understanding of Delta Lake concepts, transactional guarantees, and optimisation strategies.
- Experience using Docker across development and deployment workflows.
- Practical experience with Azure services (Functions, Key Vault, Blob Storage, ADLS Gen2).
- Experience building configuration‑driven applications.
- Strong problem‑solving skills and ability to work independently in agile environments.
- Databricks or Synapse with Delta Lake.
- CI/CD pipelines (Azure DevOps or similar) and infrastructure‑as‑code.
- Knowledge of Azure data security and governance best practices.
- Experience working in distributed or multi‑team environments.
* The salary benchmark is based on the target salaries of market leaders in their relevant sectors. It is intended to serve as a guide to help Premium Members assess open positions and to help in salary negotiations. The salary benchmark is not provided directly by the company, which could be significantly higher or lower.