Lead Data Engineer, AI & Data Pipelines Architect

Latentbridge

Greater London

On-site

GBP 90,000 - 140,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Keka Technologies Private Limited is seeking a Lead Data Engineer to design, build and maintain scalable data pipelines. You will collaborate with data scientists, software engineers and product managers to deliver robust data solutions.

The role requires 7+ years in data engineering, including leadership, and strong expertise in Apache Spark and Azure Data Factory, with hands-on cloud experience across Azure/AWS/GCP.

Qualifications

  • Bachelor's or Master's degree in Computer Science, Engineering, or related field.
  • 7+ years of experience in data engineering, with at least 3 years in a leadership role.
  • Strong proficiency in Apache Spark, Azure Data Factory, and other data pipeline tools.
  • Experience with cloud platforms such as Azure, AWS, or GCP and data governance.

Responsibilities

  • Lead the design, development, and maintenance of scalable data pipelines to support data-driven applications.
  • Collaborate with data scientists, software engineers, and product managers to understand data requirements and deliver solutions.
  • Architect and implement data ingestion, transformation, and storage using Spark, Azure Data Factory, and related tools.
  • Optimize data pipeline performance, ensuring data accuracy, reliability, and timely delivery.
  • Mentor and guide junior data engineers, promoting learning and improvement.
  • Stay up-to-date with data engineering and AI trends and apply them to projects.
  • Participate in evaluating and selecting new technologies to enhance data capabilities.
  • Contribute to AI-powered features and products leveraging machine learning concepts.

Skills

Leadership
Data engineering
Data pipelines
AI/ML concepts
Programming: Python/Scala/Java
Communication
Mentoring

Education

Bachelor's/Master's in CS/Engineering

Tools

Apache Spark
Azure Data Factory
Azure
AWS
GCP

Job description

Keka Technologies Private Limited is seeking a Lead Data Engineer to design, build and maintain scalable data pipelines. You will collaborate with data scientists, software engineers and product managers to deliver robust data solutions.

The role requires 7+ years in data engineering, including leadership, and strong expertise in Apache Spark and Azure Data Factory, with hands-on cloud experience across Azure/AWS/GCP.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Azure Data Engineer - Data Pipeline Architect
Remote Azure Data Engineer - Data Pipeline Architect

Keka Technologies Private Limited • Greater London

Remote
GBP 80,000 - 120,000
Lead Data Engineer (Consultant) - Scalable Pipelines & AI
Lead Data Engineer (Consultant) - Scalable Pipelines & AI

Kainos • Birmingham

On-site
GBP 60,000 - 80,000
Lead Data Engineer: Design, Scale & Optimize Pipelines
Lead Data Engineer: Design, Scale & Optimize Pipelines

Kainos • Greater London

On-site
GBP 70,000 - 90,000
Lead Data Engineer — AI Data Platform & Pipelines
Lead Data Engineer — AI Data Platform & Pipelines

Resume Skip • Greater London

On-site
GBP 90,000 - 130,000
Unlimited leaves
Life insurance & private health
No hierarchy
+1
Lead Data Engineer: AI-Driven Data Pipelines & Platforms
Lead Data Engineer: AI-Driven Data Pipelines & Platforms

And Digital • Birmingham

On-site
GBP 90,000 - 130,000
25 days bookable holiday + flexible BH
Pension: 6% + 2% employee
Aviva healthcare cover
+5
Lead Data Engineer: Build & Scale AI Data Pipelines
Lead Data Engineer: Build & Scale AI Data Pipelines

Datatech Analytics • City Of London

On-site
GBP 81,000 - 99,000
Lead Data Engineer: Build Scalable Data Platforms & Pipelines
Lead Data Engineer: Build Scalable Data Platforms & Pipelines

Sahaj Software • Greater London

On-site
GBP 90,000 - 120,000
Unlimited leaves
Life insurance
Private health insurance
+2
Lead Data Engineer: Architect Scalable Data Pipelines
Lead Data Engineer: Architect Scalable Data Pipelines

Kainos • Belfast City District

On-site
GBP 90,000 - 130,000
Lead Software Engineer - AI, Data Pipelines & Cloud
Lead Software Engineer - AI, Data Pipelines & Cloud

Applied Data Science Partners Ltd. • Greater London

Hybrid
GBP 100,000 - 150,000
Annual bonus and equity
26 days annual leave
BUPA Private Healthcare
+7
Lead Data Engineer: Scale Data Pipelines & Impact
Lead Data Engineer: Scale Data Pipelines & Impact

Kainos • United Kingdom

On-site
GBP 90,000 - 120,000