Cloud Data Engineer - Cloud, Python

SmartSourcing PLC; Greater London; Hybrid; GBP 44,000 - 54,000; Full time

Benefits offered by this job

Pension scheme

DDAT allowance up to £4,750

Learning & development days

Job summary

SmartSourcing PLC is recruiting a Data Engineer to join the Data, Technology and Insights unit on a 12-month fixed-term contract. The role focuses on building and operating reliable, scalable data infrastructure and pipelines in an AWS cloud environment.

You will work with data scientists, technologists and analysts to deliver data solutions, with learning opportunities and a competitive pension. Hybrid work with multiple office locations in the UK, including London and Manchester.

Qualifications

• 2+ years building end-to-end production data pipelines in a cloud environment.
• 2+ years writing production-quality Python for large, complex datasets.
• Bachelor's degree in a quantitative subject or equivalent professional experience.

Responsibilities

• Design, build and maintain scalable data pipelines to ingest, transform and process large, complex datasets.
• Develop automated web-scraping and bulk data-acquisition workflows.
• Implement data transformation, validation, testing and quality controls to ensure reliability and reproducibility.
• Build and maintain containerised data and analytics applications, including dashboards and data tools.
• Implement and manage workflow orchestration and scheduling so data pipelines are observable, reliable and maintainable.
• Apply Infrastructure as Code to support data pipelines on AWS.
• Design and maintain systems for data version control and reproducible ML workflows (MLOps).
• Collaborate with data scientists, analysts and engineers to turn requirements into data solutions.
• Contribute to a collaborative, multidisciplinary environment and share knowledge.

Skills

Python

Data engineering

Cloud

Communication

Education

BSc in a quantitative field

Tools

AWS

MLOps

Cloud/DevOps tooling

Python development

Job description

Salary: £50,000 - 50,000 per year

Requirements:

• Experience delivering production data pipelines end to end, including design, development, implementation and operational maintenance in a cloud environment.
• At least two years experience writing production-quality Python to work with complex, messy or large datasets.
• An undergraduate degree in a quantitative subject, or equivalent professional experience in data engineering, data science or cloud development.
• Strong understanding of data engineering best practices, including data modelling, transformation, validation and testing.
• Experience collaborating within multidisciplinary teams.
• Strong communication skills, including the ability to explain technical work clearly to non-technical audiences.

Responsibilities:

• Design, build and maintain scalable data pipelines to ingest, transform and process large, complex datasets.
• Develop automated web-scraping and bulk data-acquisition workflows.
• Implement data transformation, validation, testing and quality controls to make datasets reliable, reproducible and ready for analysis.
• Build and maintain containerised data and analytics applications, including dashboards and data tools.
• Implement and manage workflow orchestration and scheduling so data pipelines and jobs are observable, reliable and maintainable.
• Apply Infrastructure as Code to support data pipelines and processing workflows on AWS.
• Design and maintain systems for data version control and reproducible machine-learning workflows (MLOps).
• Work with data scientists, analysts and engineers to turn analytical and operational requirements into effective data solutions.
• Contribute to a collaborative, multidisciplinary environment and share knowledge with colleagues.

Technologies:

• AWS
• Cloud
• Support
• MLOps
• Python
• Web
• DevOps
• Machine Learning

More:

We are recruiting a Data Engineer to join our Data, Technology and Insights unit on a 12-month fixed-term contract. Youll work with data engineers, data scientists, technologists, behavioural scientists and digital forensics experts on data processing and analytical projects. This hands-on role focuses on building and operating reliable, scalable data infrastructure and pipelines in an AWS cloud environment, with opportunities for learning and professional development. The role offers a salary of £49,400 plus a DDAT allowance of up to £4,750, depending on experience, and an excellent pension. It is hybrid, with a choice of office locations in London, Manchester, Cardiff, Belfast, Edinburgh or Darlington. We offer 10 learning and development days per year.