Data Engineer - AWS, Spark & AI

Datatech Analytics; City Of London; On-site; GBP 60,000 - 75,000; Full time

Job summary

Datatech Analytics is looking for a Data Engineer to join our London team, focusing on building scalable data pipelines with AWS, Spark and AI-enabled tooling to power data platforms and analytics.

You will work with engineers, analysts and data scientists to improve data quality, reliability and performance, while exploring AI-assisted development tools such as Claude to enhance code delivery.

Qualifications

• Strong hands-on data engineering experience.
• Strong coding fundamentals across Python and SQL.
• Hands-on experience with Spark for large-scale data processing.
• Experience with AWS data services.
• Experience designing and developing scalable data pipelines.

Responsibilities

• Design, build and optimise scalable data pipelines and ETL/ELT processes.
• Develop and maintain data engineering solutions using Python, SQL and Spark.
• Work with large and complex datasets, ensuring data quality, reliability and performance.
• Use AI-assisted development tools to support coding, problem solving and delivery.
• Collaborate with engineering, analytics and data science teams to develop effective data solutions.
• Identify opportunities to automate processes and improve the scalability and efficiency of data workflows.

Skills

Python

SQL

Spark

AWS

Tools

Claude

Job description

Data Engineer - AWS, Spark & AI

Salary: £60,000-£75,000 Dependent on Experience

London: Hybrid working - 3 days per week in the office, 2 days home-based

Job Ref: J13171

Join a fast-growing technology start-up where you'll have the freedom to experiment with cutting-edge data and AI technologies, solve interesting problems and make a visible impact from day one.

Working alongside a talented team of engineers, analysts and data scientists, you won't be confined to one small part of the platform. You'll build scalable pipelines, work with large volumes of consumer data and explore new ideas at a pace that is often difficult to achieve within a larger organisation.

You'll bring strong Python, SQL and Spark fundamentals, along with genuine enthusiasm for how AI is transforming data engineering. You'll already be using AI-assisted development tools such as Claude, while having the technical ability to understand, review and challenge the code they produce.

Key Responsibilities

• Design, build and optimise scalable data pipelines and ETL/ELT processes.
• Develop and maintain data engineering solutions using Python, SQL and Spark.
• Work with large and complex datasets, ensuring data quality, reliability and performance.
• Use AI-assisted development tools to support coding, problem solving and delivery.
• Collaborate with engineering, analytics and data science teams to develop effective data solutions.
• Identify opportunities to automate processes and improve the scalability and efficiency of data workflows.

Essential Skills and Experience

• Strong hands-on data engineering experience.
• Strong coding fundamentals across Python and SQL.
• Hands-on experience working with Spark for large-scale data processing.
• Hands-on experience working with AWS data services.
• Experience designing and developing scalable data pipelines and ETL/ELT processes.
• Practical experience using AI-assisted development tools such as Claude within your day-to-day work.
• Strong communication skills and the ability to work effectively within a fast-moving, collaborative environment.

If you're a Data Engineer with strong technical fundamentals who wants the freedom to innovate, explore how AI is changing engineering and help shape a data platform as the business scales, we want to hear from you.

Please note we can only accept applications from those with current UK working rights for this role. This client cannot offer visa sponsorship.

www.datatech.org.uk