Data Engineer AWS

Athsai; Remote; GBP 60,000 - 80,000; Full time

Job summary

A leading data solutions provider is seeking a Senior Data Engineer to design and optimize scalable data pipelines using PySpark and AWS. The ideal candidate will have over 6 years of experience in data engineering, strong skills in Python and SQL, and expertise in ETL processes. This remote role offers the chance to work on high-impact data platforms and opportunities for career growth within a collaborative culture.

Benefits

Competitive compensation

Benefits package

Collaborative culture

Qualifications

• 6+ years of hands-on experience in data engineering or related roles.
• Strong expertise in Python, PySpark, and SQL with experience in writing optimized, production-grade code.
• Proven experience designing and implementing ETL pipelines for large-scale data platforms.

Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark and Python.
• Develop and manage ETL / ELT workflows, ensuring data accuracy.
• Architect and implement cloud-native data solutions on AWS.

Skills

Python

PySpark

SQL

Apache Spark

AWS

Terraform

CI/CD

Tools

GitHub Actions

Apache Airflow

Docker

Kubernetes

Job description

We Are Hiring – Senior Data Engineer (PySpark & AWS)

Salary: 300 outside IR35

Location : Remote in UK

We are looking for an experienced and highly skilled Senior Data Engineer to join our growing data engineering team. This role is ideal for a passionate engineer who thrives in building scalable data platforms, designing robust pipelines, and working with cutting-edge cloud technologies.

About The Role

As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing large-scale data pipelines that power analytics, reporting, and machine learning initiatives. You will work closely with data scientists, analysts, and platform teams to ensure data is reliable, secure, and available in real time and batch processing environments.

Key Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark and Python for high-volume, high-velocity data processing.
• Develop and manage ETL / ELT workflows, ensuring data accuracy, consistency, and performance.
• Orchestrate complex workflows using Apache Airflow, including scheduling, dependency management, and failure handling.
• Architect and implement cloud-native data solutions on AWS, following best practices for performance, scalability, and security.
• Work extensively with AWS services such as API Gateway, AWS Lambda, Amazon Redshift, AWS Glue, Amazon CloudWatch, Amazon S3, EMR, and IAM.
• Use Terraform to provision and manage AWS infrastructure as code, ensuring reproducible and reliable environments.
• Build and maintain CI / CD pipelines using GitHub Actions to automate testing, deployment, and infrastructure changes.
• Optimize Spark jobs, tune performance, and troubleshoot production issues across distributed systems.
• Collaborate with cross-functional teams to define data architecture, governance, and best practices.

Required Qualifications

• 6+ years of hands‑on experience in data engineering or related roles.
• Strong expertise in Python, PySpark, and SQL with experience in writing optimized, production‑grade code.
• In‑depth knowledge of Apache Spark internals and Apache Airflow.
• Proven experience designing and implementing ETL pipelines for large‑scale data platforms.
• Strong hands‑on experience with AWS cloud services, especially API Gateway, Lambda, Redshift, Glue, CloudWatch, S3, and EMR.
• Experience provisioning infrastructure using Terraform.
• Practical experience building CI / CD pipelines using GitHub Actions.

Preferred Qualifications

• Experience with real‑time data streaming using Kafka, Kinesis, or similar technologies.
• Familiarity with containerization tools such as Docker and Kubernetes.
• Knowledge of data governance, data quality frameworks, and monitoring strategies.

Why Join Us?

• Work on large-scale, high-impact data platforms.
• Opportunity to shape modern data architecture in a cloud-first environment.
• Collaborative, innovative, and growth-focused culture.
• Competitive compensation and benefits.