Data Engineer AWS

Athsai; Remote; GBP 26,000 - 34,000; Full time

Job summary

A leading technology company in the UK is seeking a Senior Data Engineer to design and optimize scalable data pipelines using PySpark and AWS. The role involves working with cloud-native data solutions, developing ETL workflows, and collaborating with cross-functional teams to enhance data architecture. Ideal candidates will have over 6 years of experience in data engineering, strong expertise in Python and SQL, and familiarity with tools like Apache Airflow and Terraform. Competitive compensation and a collaborative culture are offered.

Benefits

Competitive compensation

Growth-focused culture

Collaborative work environment

Qualifications

• 6+ years of hands-on experience in data engineering or related roles.
• Strong expertise in Python, PySpark, and SQL.
• In-depth knowledge of Apache Spark internals and Apache Airflow.

Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark and Python.
• Develop and manage ETL / ELT workflows.
• Architect and implement cloud-native data solutions on AWS.

Skills

Data pipeline design

PySpark

AWS services

ETL development

CI/CD pipelines

Python

SQL

Tools

Apache Airflow

Terraform

GitHub Actions

Docker

Kubernetes

Kafka

Amazon Redshift

Job description

We Are Hiring – Senior Data Engineer (PySpark & AWS)

Salary: 300 outside IR35

Location : Remote in UK

We are looking for an experienced and highly skilled Senior Data Engineer to join our growing data engineering team. This role is ideal for a passionate engineer who thrives in building scalable data platforms, designing robust pipelines, and working with cutting-edge cloud technologies.

About The Role

As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing large-scale data pipelines that power analytics, reporting, and machine learning initiatives. You will work closely with data scientists, analysts, and platform teams to ensure data is reliable, secure, and available in real time and batch processing environments.

Key Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark and Python for high-volume, high-velocity data processing.
• Develop and manage ETL / ELT workflows, ensuring data accuracy, consistency, and performance.
• Orchestrate complex workflows using Apache Airflow, including scheduling, dependency management, and failure handling.
• Architect and implement cloud-native data solutions on AWS, following best practices for performance, scalability, and security.
• Work extensively with AWS services such as API Gateway, AWS Lambda, Amazon Redshift, AWS Glue, Amazon CloudWatch, Amazon S3, EMR, and IAM.
• Use Terraform to provision and manage AWS infrastructure as code, ensuring reproducible and reliable environments.
• Build and maintain CI / CD pipelines using GitHub Actions to automate testing, deployment, and infrastructure changes.
• Optimize Spark jobs, tune performance, and troubleshoot production issues across distributed systems.
• Collaborate with cross-functional teams to define data architecture, governance, and best practices.

Required Qualifications

• 6+ years of hands‑on experience in data engineering or related roles.
• Strong expertise in Python, PySpark, and SQL with experience in writing optimized, production‑grade code.
• In‑depth knowledge of Apache Spark internals and Apache Airflow.
• Proven experience designing and implementing ETL pipelines for large‑scale data platforms.
• Strong hands‑on experience with AWS cloud services, especially API Gateway, Lambda, Redshift, Glue, CloudWatch, S3, and EMR.
• Experience provisioning infrastructure using Terraform.
• Practical experience building CI / CD pipelines using GitHub Actions.

Preferred Qualifications

• Experience with real‑time data streaming using Kafka, Kinesis, or similar technologies.
• Familiarity with containerization tools such as Docker and Kubernetes.
• Knowledge of data governance, data quality frameworks, and monitoring strategies.

Why Join Us?

• Work on large-scale, high-impact data platforms.
• Opportunity to shape modern data architecture in a cloud-first environment.
• Collaborative, innovative, and growth-focused culture.
• Competitive compensation and benefits.