Hybrid Data Engineer: AWS, Python & Pipelines

SmartSourcing Ltd; Greater London; Hybrid; GBP 45,000 - 54,000; Full time

Benefits offered by this job

Hybrid working

Pension scheme

Learning & development days (10 per yr

Office locations London/UK

Job summary

SmartSourcing Ltd is seeking a Data Engineer to design, build and operate scalable data pipelines in an AWS cloud environment. You will write production-quality Python, collaborate with data scientists and engineers, and contribute to data modelling, validation and reproducibility across complex datasets.

The role is hands-on, with opportunities to develop end-to-end data infrastructure, automate workflows and deliver analytics tools, on a hybrid basis with learning and development support.

Qualifications

• Experience delivering production data pipelines end-to-end in a cloud environment.
• Experience writing production-quality Python to work with large datasets (minimum 2 years).
• Undergraduate degree in a quantitative subject, or equivalent professional experience in data engineering, data science or cloud development.
• Strong understanding of data engineering best practices including data modelling, transformation, validation and testing.
• Experience working collaboratively within multidisciplinary teams.
• Strong communication skills with ability to explain technical work to non-technical audiences.

Responsibilities

• Help build and operate reliable, scalable data infrastructure and pipelines across an AWS cloud environment.
• Produce high-quality Python code and other languages as needed.
• Design, build and maintain scalable data pipelines to ingest, transform and process large datasets.
• Develop automated web scraping and bulk data acquisition workflows.
• Implement data transformation, validation, testing and quality controls for reliable datasets.
• Build and maintain containerised data and analytics apps, including dashboards and tools.
• Implement and manage workflow orchestration and scheduling for observability and reliability.
• Apply Infrastructure as Code (IaC) to support pipelines on AWS.
• Design and maintain data version control and reproducible MLOps workflows.
• Work with data scientists and engineers to translate requirements into solutions.
• Share knowledge in a collaborative multidisciplinary environment.

Skills

Data pipelines end-to-end

Python production-quality code

Cloud data engineering

Data modelling

Data transformation

Data validation & testing

Cross-functional collaboration

Strong communication

Education

Undergraduate degree in quantitative subject

Tools

Python

AWS

Cloud environments

Job description

SmartSourcing Ltd is seeking a Data Engineer to design, build and operate scalable data pipelines in an AWS cloud environment. You will write production-quality Python, collaborate with data scientists and engineers, and contribute to data modelling, validation and reproducibility across complex datasets.

The role is hands-on, with opportunities to develop end-to-end data infrastructure, automate workflows and deliver analytics tools, on a hybrid basis with learning and development support.