Principal Data Engineer

Lloyds Banking Group; Hyderabad; Hybrid; INR 4,200,000 - 6,600,000; Full time

Benefits offered by this job

Hybrid work

Job summary

Lloyds Banking Group plays a pivotal role through its Lloyds Technology Centre in Hyderabad, seeking an Engineering Leader – Data to own end-to-end data product delivery across multiple squads. The role involves shaping data platforms, defining semantic models and KPIs, and driving AI-ready capabilities with scalable, secure architecture.

You will lead teams, coach high‑performing engineers, and partner with the Product Owner to plan roadmaps, while ensuring reliability, compliance, and value

Qualifications

• Leadership ownership of complex data platform deliveries across multiple squads.
• Experience defining semantic layers, KPI logic, and BI consumption models.
• Understanding AI/ML data needs, feature engineering, and knowledge graphs.
• Deep modelling skills: Dimensional/Star-Snowflake, Data Vault 2.0.
• Proficiency in cloud data platforms and performance trade-offs.
• Experience with streaming systems and event pipelines (Kafka/Pub/Sub/Kinesis).
• Strong software engineering practices: CI/CD, code reviews, testing.
• Infrastructure as code and cloud runtimes (Terraform/Docker/Kubernetes).
• Knowledge governance, data contracts, and lineage practices.
• Finance ops and observability for cloud data platforms.

Responsibilities

• Delivery ownership: define roadmaps, OKRs, releases; ensure on-time delivery.
• Team ownership: hiring, coaching, performance management, standards.
• Establish scalable data engineering patterns and reusable components.
• Define enterprise semantic models, KPI definitions, and BI-ready views.
• Shape data for ML/GenAI with knowledge graphs and embeddings.
• Guide dimensional modelling and Data Vault 2.0 standards.
• Oversee streaming pipelines with exactly-once semantics and SLAs.
• Ensure data quality, metadata capture, and end-to-end lineage.
• Implement observability, cost/performance guardrails for cloud DW.
• Ensure security, IAM, encryption, and governance across pipelines.

Skills

Leadership

Semantic layers

BI expertise

AI-ready design

Modelling depth

Cloud data platforms

Streaming systems

Software-engineering mindset

IaC & runtime

Data quality & governance

FinOps & observability

Tools

BigQuery

Snowflake

Redshift

Synapse

Databricks

Kafka

Spark

Flink

Terraform

Docker

Kubernetes

Job description

End Date

Tuesday 29 September 2026

We Support Flexible Working – Click here for more information on flexible working options

Flexible Working Options

Hybrid Working

Job Description Summary

The Data Engineer will work alongside the Team Product Owner in a Feature Team managing delivery by leading others. They will operate as co-leader with the Team PO with involvement in decision making and planning of Product roadmap. In some cases, this role will be an Individual Contributor operating at Lab rather than feature team level.

Job Description

Job Title: Engineering Leader – Data (Lloyds Technology Centre)
Location: Hyderabad, India (Hybrid: at least 2 days/week in office)
Experience: 15+ years

About Lloyds Technology Centre

Lloyds Technology Centre is the strategic technology hub for Lloyds Banking Group , enabling digital transformation and innovation across the banking ecosystem. Our mission is to build secure, scalable, and data-driven solutions that empower millions of customers and ensure compliance with financial regulations.

Role Overview

Own the delivery and operation of data products and platforms across multiple squads, with end‑to‑end accountability for outcomes, quality, and reliability. Establish reusable, metadata ‑ driven engineering patterns; elevate semantic models, KPI/metrics , and BI consumption ; and build an AI ‑ ready foundation (knowledge layers, knowledge graphs, semantic models) that accelerates analytics and machine learning use‑cases across the organisation.

Key Responsibilities

• Delivery ownership: Define roadmaps, OKRs, and release plans; oversee scope, estimation, risk, and stakeholder communication; ensure on‑time, on‑budget, quality delivery across squads.

• Team ownership & leadership: Build and lead high‑performing teams (hiring, coaching, performance management); set coding standards, DoR/DoD, working agreements, and succession plans.

• Engineering patterns at scale: Establish factory‑mode delivery via inner‑sourced, reusable frameworks (ingestion, transform, quality, lineage, observability) and golden paths with strong documentation

• Semantic layer & metrics: Define and govern enterprise semantic models, KPI/metric definitions , conformed dimensions, metric stores, and query‑ready views to enable consistent BI and self‑serve analytics.

• AI readiness & knowledge layers: Shape data for ML/GenAI—ontologies, knowledge graphs , feature/embedding strategies, and patterns that make the platform AI‑ready by design.

• Data modelling & performance: Guide dimensional (star/snowflake) and Data Vault 2.0 modelling; set standards for physical design on cloud warehouses (partitioning, clustering, caching, workload mgmt.)

• Streaming & real ‑ time: Oversee event pipelines (Kafka/Pub/Sub/Kinesis + Flink/Spark) with exactly‑once semantics, replay, SLAs/SLOs, and resiliency patterns.

• Quality, metadata & lineage: Make quality the default—data contracts, DQ rules/tests, reconciliation—and automate capture/propagation of technical/business metadata and end‑to‑end lineage

• Observability & FinOps: Implement platform and pipeline telemetry (logs, metrics, tracing) and proactive cost/performance guardrails for cloud DW/compute; continuously optimise jobs/queries.

• Security & compliance: Champion IAM, least‑privilege patterns, encryption, secrets mgmt., and auditability; embed privacy‑by‑design and regulatory controls into pipelines and platforms.

• Vendor & partner management: Govern partner delivery (e.g., TCS/HCL), enforce standards, and ensure reusable IP is contributed back to inner‑source repos.

Required Skills

• Leadership & ownership: Proven record of owning complex data platform deliveries, leading multiple squads, and managing outcomes/SLAs in production.

• Semantic & BI expertise: Hands‑on experience defining semantic layers , KPI/metric logic, metric stores, and consumption models that scale across BI tools.

• AI‑ready design: Practical understanding of AI/ML data needs—feature engineering, data products for ML, knowledge graphs/ontologies , vector/embedding patterns—and how to operationalise them on cloud platforms.

• Modelling depth: Dimensional (star/snowflake), Data Vault 2.0, SCDs, schema evolution/versioning; ability to set and enforce modelling standards.

• Cloud data platforms: Strong experience with one or more of BigQuery, Snowflake, Redshift, Synapse/Databricks SQL; clarity on cloud DW vs traditional MPP trade‑offs and performance levers.

• Streaming systems: Kafka/Pub/Sub/Kinesis and Spark/Flink; event schema management (Avro/Protobuf), idempotency, back‑pressure, state mgmt.

• Software‑engineering mindset: Git‑based workflows, code reviews, automated testing, CI/CD, artifact versioning; inner‑sourcing culture and documentation excellence.

• IaC & runtime: Terraform/CloudFormation, Docker/Kubernetes; sizing, autoscaling, job concurrency, and reliability engineering.

• Data quality & governance: Data contracts; testing frameworks (e.g., Great Expectations/dbt tests); catalogue/lineage tooling; policy enforcement via pipelines.

• FinOps & observability: Cost modelling for cloud DW/compute, workload tuning, budgets/alerts; platform and pipeline telemetry with actionable SLOs.