AWS Data Engineer - PySpark/AI

Datatech Analytics; City Of London; Hybrid; GBP 45,000 - 75,000; Full time

Job summary

Datatech Analytics is seeking an AWS Data Engineer to own and evolve data products in a hybrid London-based role. You will design and optimise ETL/ELT pipelines, expand the AWS data lake, and partner with product and engineering to structure data for scalable analytics.

You will work with large event data, enhance data quality, and contribute to CI/CD improvements while exploring new technologies and Claude Code-enabled workflows.

Qualifications

• Strong SQL for transforming, cleaning and loading data.
• Commercial Python, PySpark and Pandas experience.
• Experience with orchestration tools such as Airflow, Dagster or Celery.
• Practical experience with Spark and EMR Hands-on knowledge of AWS services, including Lambda, RDS and S3.
• Experience using Claude Code as part of your day-to-day engineering workflow.
• An understanding of CI/CD and DataOps practices.
• Strong communication skills and the ability to collaborate with colleagues internationally.

Responsibilities

• Build and improve ETL and ELT pipelines supporting analytics and customer-facing products.
• Expand and optimise the existing AWS data lake and architecture.
• Work closely with product and engineering teams to shape how data is structured and used.
• Improve data quality, accessibility and reliability across the platform.
• Automate manual processes and strengthen CI/CD across data workflows.
• Explore new technologies and influence future engineering decisions.
• Use Claude Code to support development, debugging, testing and documentation.

Skills

SQL

Python

PySpark

Pandas

Claude Code

DataOps

CI/CD

Communication

Cross-team collaboration

Tools

Airflow

Dagster

Celery

Spark

EMR

Lambda

S3

RDS

Job description

AWS Data Engineer - PySpark/AI

Location: Holborn, London - hybrid, three office days per week

Salary: Up to £75,000 dependent on experience

Ref: J13171

Our client is a growing technology business developing data and AI products that help organisations gain deeper insight into consumer behaviour. They are expanding their engineering team and looking for a Data Engineer who wants to take ownership, contribute ideas and have a visible impact on how the platform develops.

You’ll join a small, collaborative team working with large volumes of complex event data. From improving the underlying architecture to developing dependable data products, you’ll help keep the journey from raw data to customer insight fast, scalable and trustworthy.

You’ll have the opportunity to:

• Build and improve ETL and ELT pipelines supporting analytics and customer-facing products
• Expand and optimise the existing AWS data lake and architecture
• Work closely with product and engineering teams to shape how data is structured and used
• Improve data quality, accessibility and reliability across the platform
• Automate manual processes and strengthen CI/CD across data workflows
• Explore new technologies and influence future engineering decisions
• Use Claude Code to support development, debugging, testing and documentation

Experience required:

• Strong SQL skills for transforming, cleaning and loading data
• Commercial Python, PySpark and Pandas experience
• Experience with orchestration tools such as Airflow, Dagster or Celery
• Practical experience with Spark and EMRHands-on knowledge of AWS services, including Lambda, RDS and S3
• Experience using Claude Code as part of your day-to-day engineering workflow
• An understanding of CI/CD and DataOps practices
• Strong communication skills and the ability to collaborate with colleagues internationally

Experience with LLMs, fine-tuning, RAG or statistics would be useful but is not essential.

This is an opportunity to join an ambitious business at an exciting stage of its growth, take genuine ownership and help develop data products that customers can trust and rely on.