Lead Data Engineer Python, Spark, AWS , ETL pipelines

Athsai; Greater London; Remote; GBP 90,000; Full time

Job summary

An established industry player is seeking a Lead Data Engineer to join their dynamic team in London. This exciting remote role requires a dedicated professional with substantial experience in big data technologies and strong programming skills in Python, Scala, and SQL. You will work closely with stakeholders to enhance data quality and implement robust data pipelines using AWS services. Your expertise in ETL tools like Apache Kafka, Spark, and Airflow will be invaluable as you contribute to meaningful advancements in data initiatives. If you're passionate about leveraging data to drive insights, this is the perfect opportunity for you.

Qualifications

• Substantial experience in big data technologies is essential.
• Strong programming skills in Python, Scala, and SQL are required.

Responsibilities

• Enhance data quality across projects by collaborating with stakeholders.
• Implement data pipelines utilizing cloud-native services on AWS.

Skills

ETL Tools

Apache Kafka

Spark

Airflow

Python

Scala

SQL

AWS Services

Data Pipeline Implementation

Job description

Get AI-powered advice on this job and more exclusive features.

Lead Data Engineer

Job Location: London (Remote)

Eligibility for SC clearance is required.

Permanent role - 90k GBP

We are looking for a dedicated professional with substantial experience in big data technologies to join our team. Your collaborative skills will be key as you work alongside stakeholders, data architects, and data scientists to enhance data quality across our projects.

We value your expertise in ETL tools, including Apache Kafka, Spark, and Airflow, and your strong programming skills in Python, Scala, and SQL will further enrich our capabilities. Additionally, your experience in implementing data pipelines utilizing cloud-native services on AWS will be invaluable.

Your comprehensive knowledge of AWS services such as API Gateway, Lambda, Redshift, Glue, and CloudWatch will enable us to optimize our data processes.

We look forward to your contributions in making meaningful advancements in our customers' data initiatives.

Responsibilities

• Are you currently holding SC clearance or can you become eligible to obtain it? Your clearance is essential for the role.
• If you are a qualified Lead AWS Data Engineer/Architect, please contact me.

Seniority Level

Associate

Employment Type

Full-time

Job Function

Information Technology, Analyst, and Consulting

Industries

IT Services and IT Consulting, IT System Data Services, and Data Infrastructure and Analytics