Data Engineer - AWS, PySpark & AI Impact

Datatech Analytics; City of Westminster; On-site; GBP 60,000 - 90,000; Full time

Job summary

Datatech Analytics is seeking a Data Engineer to own and shape data platform evolution. You’ll join a small, collaborative team handling large volumes of complex event data and delivering reliable data products at scale.

You’ll design and optimize ETL/ELT pipelines, expand the AWS data lake, and collaborate with product and engineering teams to structure data for analytics while improving quality and CI/CD across workflows.

Qualifications

• Strong SQL skills for transforming, cleaning and loading data.
• Commercial Python, PySpark and Pandas experience.
• Experience with orchestration tools such as Airflow, Dagster or Celery.
• Practical experience with Spark and EMR.
• Hands-on knowledge of AWS services, including Lambda, RDS and S3.
• Experience using Claude Code as part of your day-to-day engineering workflow.
• An understanding of CI/CD and DataOps practices.
• Strong communication skills and the ability to collaborate with colleagues internationally.

Responsibilities

• Build and improve ETL and ELT pipelines supporting analytics and customer-facing products
• Expand and optimise the existing AWS data lake and architecture
• Work closely with product and engineering teams to shape how data is structured and used
• Improve data quality, accessibility and reliability across the platform
• Automate manual processes and strengthen CI/CD across data workflows
• Explore new technologies and influence future engineering decisions
• Use Claude Code to support development, debugging, testing and documentation

Skills

SQL

Python/PySpark/Pandas

Airflow/Dagster/Celery

Spark/EMR

AWS (Lambda/RDS/S3)

Claude Code

CI/CD & DataOps

Communication

Tools

Airflow

Dagster

Celery

Spark

EMR

Claude Code

Job description

Datatech Analytics is seeking a Data Engineer to own and shape data platform evolution. You’ll join a small, collaborative team handling large volumes of complex event data and delivering reliable data products at scale.

You’ll design and optimize ETL/ELT pipelines, expand the AWS data lake, and collaborate with product and engineering teams to structure data for analytics while improving quality and CI/CD across workflows.