Data Engineer

SmartSourcing Ltd; Greater London; Hybrid; GBP 45,000 - 54,000; Full time

Benefits offered by this job

Hybrid working

Pension scheme

Learning & development days (10 per yr

Office locations London/UK

Job summary

SmartSourcing Ltd is seeking a Data Engineer to design, build and operate scalable data pipelines in an AWS cloud environment. You will write production-quality Python, collaborate with data scientists and engineers, and contribute to data modelling, validation and reproducibility across complex datasets.

The role is hands-on, with opportunities to develop end-to-end data infrastructure, automate workflows and deliver analytics tools, on a hybrid basis with learning and development support.

Qualifications

• Experience delivering production data pipelines end-to-end in a cloud environment.
• Experience writing production-quality Python to work with large datasets (minimum 2 years).
• Undergraduate degree in a quantitative subject, or equivalent professional experience in data engineering, data science or cloud development.
• Strong understanding of data engineering best practices including data modelling, transformation, validation and testing.
• Experience working collaboratively within multidisciplinary teams.
• Strong communication skills with ability to explain technical work to non-technical audiences.

Responsibilities

• Help build and operate reliable, scalable data infrastructure and pipelines across an AWS cloud environment.
• Produce high-quality Python code and other languages as needed.
• Design, build and maintain scalable data pipelines to ingest, transform and process large datasets.
• Develop automated web scraping and bulk data acquisition workflows.
• Implement data transformation, validation, testing and quality controls for reliable datasets.
• Build and maintain containerised data and analytics apps, including dashboards and tools.
• Implement and manage workflow orchestration and scheduling for observability and reliability.
• Apply Infrastructure as Code (IaC) to support pipelines on AWS.
• Design and maintain data version control and reproducible MLOps workflows.
• Work with data scientists and engineers to translate requirements into solutions.
• Share knowledge in a collaborative multidisciplinary environment.

Skills

Data pipelines end-to-end

Python production-quality code

Cloud data engineering

Data modelling

Data transformation

Data validation & testing

Cross-functional collaboration

Strong communication

Education

Undergraduate degree in quantitative subject

Tools

Python

AWS

Cloud environments

Job description

Salary: £45,000 - 54,150 per year

Requirements

• Experience delivering production data pipelines end-to-end, including design, development, implementation and operational maintenance in a cloud environment.
• Experience writing production-quality Python to work with complex, messy or large datasets (minimum 2 years).
• An undergraduate degree in a quantitative subject, or equivalent professional experience in data engineering, data science or cloud development.
• A strong understanding of data engineering best practices, including data modelling, transformation, validation and testing.
• Experience working collaboratively within multidisciplinary teams.
• Strong communication skills, with the ability to explain technical work clearly and engage with non-technical audiences.

Responsibilities

• Help build and operate reliable, scalable data infrastructure and pipelines across an AWS cloud environment.
• Produce high-quality code in Python and other languages as needed.
• Design, build and maintain scalable data pipelines to ingest, transform and process large and complex datasets.
• Develop automated web scraping and bulk data acquisition workflows.
• Implement data transformation, validation, testing and quality controls to ensure datasets are reliable, reproducible and ready for analysis.
• Build and maintain containerised data and analytics applications, including dashboards and data tools.
• Implement and manage workflow orchestration and scheduling to ensure data pipelines and jobs are observable, reliable and maintainable.
• Apply Infrastructure as Code (IaC) to support data pipelines and processing workflows on AWS.
• Design and maintain systems for data version control and reproducible machine learning workflows (MLOps).
• Work closely with data scientists, analysts and engineers to translate analytical and operational requirements into effective data solutions.
• Contribute to a collaborative, multidisciplinary environment and share knowledge with colleagues.

Technologies

• AWS
• Cloud
• Support
• Machine Learning
• MLOps
• Python
• Web

More

We are a Government client looking for a Data Engineer to join our Data, Technology and Insights (DTI) unit on a 12-month fixed-term contract. You will join a multidisciplinary team of data engineers, data scientists, technologists, behavioural scientists and digital forensics experts, working on data processing and analytical projects for both internal team and user-initiated projects. This is a hands‑on role with plenty of opportunity for learning and professional development. We offer a salary of £49,400 plus a DDAT allowance of up to £4,750 depending on experience, an excellent pension, hybrid working, choice of office location in London, Manchester, Cardiff, Belfast, Edinburgh or Darlington, and 10 learning and development days per year. We work on diverse projects with genuine public impact alongside experienced data, technology and analytical specialists.

last updated 38 week of 2026