Data Engineer

Qualient Technology Solutions UK Limited; City Of London; On-site; GBP 50,000 - 70,000; Full time

Job summary

A leading technology firm in the UK is seeking a Data Engineer to design and deploy production data pipelines in a big data environment. Candidates should have experience with AWS cloud technologies and proficiency in Java, Python, and SQL. Ideal applicants must exhibit strong problem-solving skills and thrive in collaborative, fast-paced settings.

Qualifications

• Design and deploy production data pipelines within a big data architecture.
• Experience with AWS cloud technologies is essential.
• Ability to integrate data from multiple sources.

Skills

Problem-solving

Analytical thinking

Collaboration

Java

Python

Scala

Spark

SQL

Tools

AWS

EMR

Glue

RedShift

Kinesis

Lambda

DynamoDB

Job description

Overview

Our tech stack

AWS is a significant growth area for us with a diverse and growing capability and we are looking for a Data Engineer with experience in AWS cloud technologies for ETL pipeline, data warehouse and data lake design/building and data movement.

• AWS data and analytics services (or open-source equivalent) such as EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB.

Essential requirements

• You thrive in problem-solving and analytical thinking
• You enjoy collaborating with multiple stakeholders in a fast-paced environment
• Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL.
• Experience performing tasks such as writing scripts, extracting data using APIs, writing SQL queries etc.
• Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources through ingestion and curation functions on AWS cloud using AWS native or custom programming