Lead Data Engineer

Placements24; London Borough of Newham; Hybrid; GBP 90,000 - 140,000; Full time

Benefits offered by this job

Hybrid work arrangement

Job summary

Placements24 is recruiting a Lead Data Engineer in East London to lead the design, development, and maintenance of scalable data pipelines and data platforms. The role blends hands-on engineering with team leadership in a hybrid work setting.

The candidate should have strong experience in data warehousing, ETL/ELT processes, and cloud data services, with a track record of delivering robust data solutions in e-commerce environments.

Qualifications

• Bachelor's degree in CS/Engineering or related field.
• 5+ years in data engineering; 2+ years in lead role.
• Experience designing ETL/ELT pipelines with Spark or Kafka.
• Strong cloud data services experience (AWS/Azure/GCP).
• Proficiency in SQL and Python/Scala.
• Familiarity with data warehousing concepts and dimensional modeling.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines.
• Architect data warehouses and data lakes on cloud platforms.
• Optimize data infrastructure for performance and cost.
• Lead and mentor a team of data engineers.
• Ensure data governance, quality, and security.

Skills

Data pipelines

Spark/Kafka

SQL/NoSQL

Python/Scala

Cloud platforms

Leadership

Education

Bachelor's in CS/Engineering

Master's preferred

Tools

AWS

Azure

GCP

Redshift/Snowflake

Job description

About the Role

Our client, a prominent e-commerce platform, is seeking an experienced Lead Data Engineer to join their growing data team in East London. This role is responsible for designing, building, and maintaining scalable data pipelines and infrastructure that support the company's analytics and machine learning initiatives. You will lead a team of data engineers, architect data solutions, and ensure the efficiency and reliability of the data platform. The ideal candidate has a strong background in data warehousing, ETL processes, big data technologies, and cloud data services. This position offers a hybrid work arrangement, combining remote flexibility with essential in-office collaboration in East London.

Key Responsibilities

• Design, develop, and maintain robust and scalable ETL/ELT pipelines to ingest and process large volumes of data.
• Architect and build data warehouses and data lakes on cloud platforms (e.g., AWS , Azure , GCP ).
• Optimize data infrastructure for performance, scalability, and cost-efficiency.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.
• Implement data governance and data quality best practices.
• Lead and mentor a team of data engineers, providing technical guidance and support.
• Monitor and troubleshoot data pipelines and infrastructure issues.
• Evaluate and implement new data technologies and tools.
• Ensure data security and compliance with relevant regulations.
• Automate data processes and workflows using scripting and programming languages.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related quantitative field; Master's degree is a plus.
• 5+ years of experience in data engineering, with at least 2 years in a lead or senior role.
• Proven experience designing and building data pipelines using tools like Apache Spark , Kafka , or similar.
• Strong experience with cloud data services (e.g., AWS Redshift, S3, Glue; Azure Data Factory, SQL Data Warehouse; Google BigQuery, Cloud Storage).
• Proficiency in SQL and experience with relational and NoSQL databases.
• Experience with programming languages such as Python or Scala.
• Familiarity with data warehousing concepts and dimensional modeling.
• Excellent problem-solving, analytical, and communication skills.
• Demonstrated leadership and team management capabilities.
• Experience working in an e-commerce environment is a significant advantage.
• Ability to work a hybrid schedule from our offices in East London.

Benefits

Our client offers a competitive salary, comprehensive health and wellness benefits, performance bonuses, and opportunities for continuous learning and professional development. This hybrid role provides flexibility while fostering team collaboration. Join a data-driven organization in East London that values innovation and offers a challenging and rewarding career path in the exciting field of big data.