Data Engineer

W Talent; Remote; GBP 80,000 - 84,000; Full time

Job summary

A leading technology recruitment firm is seeking an experienced Data Engineer for a fully remote position with occasional site visits to London. The ideal candidate will have at least 8 years of hands-on experience in data engineering, proficiency in designing and debugging ETL pipelines, and expertise in Python, PySpark, SQL, and AWS services. The salary for this role is competitive, ranging from £80,000 to £84,000, and candidates must be eligible for SC Clearance.

Qualifications

• Minimum of 8 years of hands-on experience in data engineering projects.
• Proficient in designing, implementing, and debugging ETL pipelines.
• Expertise in Python, PySpark, and SQL is essential.
• Experience with Apache Spark and Airflow is required.
• Familiar with AWS cloud services and designing data pipelines.

Skills

Data Engineering

ETL pipeline design

Python

PySpark

SQL

Apache Spark

Airflow

AWS

Terraform

GitHub Actions

Job description

Excellent opportunity for an experienced Data Engineer looking for a fully remote role, ideally you will have at least 5 years experience.

You will need to be eligible for SC Clearance for this role

Data Engineer

Location - Fully Remote - occasional site visits to London

Salary - £80,000 - £84,000

Key Skills:

• Data Engineering: ETL pipeline design, implementation, and debugging
• Programming: Python, PySpark, SQL
• Big Data & Workflow: Apache Spark, Airflow
• Cloud Services: AWS (API Gateway, Lambda, Redshift, Glue, CloudWatch)
• Infrastructure as Code: Terraform
• CI/CD: GitHub Actions

Key Experience

• At least 8 years of hands on experience in data engineering projects
• Good hands on experience of designing, implementing, debugging ETL pipeline
• Expertise in Python, pySpark and SQL languages in must
• Expertise with Spark and Airflow
• Experience of designing data pipeline using cloud native services on AWS
• Extensive knowledge of AWS services like API Gateway, Lambda, Redshift, Glue, Cloudwatch, etc.
• Iac experience of deploying AWS resources using terraform