Data Engineering Lead (GCP, Bigquery, ETL, Prompt Engineering) Associate Director

HSBC; Hyderabad; On-site; INR 400,000 - 700,000; Full time

Job summary

HSBC Software Development India seeks an Associate Director, Data and Analytics Specialist to own end-to-end data platform architecture and batch/streaming pipelines, defining patterns, standards and reference designs in line with HSBC principles.

You will lead hands-on delivery using GCP services (BigQuery, Dataflow, Pub/Sub, Airflow), drive data modelling, governance, CI/CD, and FinOps, and coach a team of data engineers in an agile environment to meet business objectives.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field; Master’s preferred.
• Significant experience in data engineering and cloud platform delivery, including end-to-end solution design.
• Strong hands-on experience with GCP data/platform services (BigQuery, Dataflow, Pub/Sub, Composer/Airflow, Dataproc, Cloud Storage).
• Proven expertise in data modelling and designing performant analytical data structures (dimensional modelling; data vault/lakehouse).
• Experience leading engineers, coaching, and delivering in an agile environment.

Responsibilities

• Own end-to-end data platform architecture and pipelines (batch/streaming).
• Provide hands-on technical leadership across GCP services and ELT/ETL development.
• Define patterns, standards, and reference designs aligned to HSBC principles.
• Mentor data engineers and collaborate with product owners and stakeholders.

Skills

Data engineering

GCP

ETL/ELT

Leadership

Data modelling

Education

Bachelor's degree in CS or related field

Master's degree preferred

Tools

BigQuery

Dataflow

Pub/Sub

Airflow/Composer

Dataproc

Job description

Some careers shine brighter than others.
If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.
HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions.
We are currently seeking an experienced professional to join our team in the role of Associate Director, Data and Analytics Specialist

In This Role, You Will

• Own end-to-end data platform and pipeline architecture (batch/streaming), defining patterns, standards, and reference designs aligned to HSBC engineering and risk principles.
• Provide hands‑on technical leadership across core GCP services such as BigQuery, Cloud Storage, Pub/Sub, Dataflow, Dataproc, Composer (Airflow), Cloud Run/GKE, IAM, Cloud KMS, VPC, and related tooling. Lead development of resilient ELT/ETL pipelines, implementing orchestration, scheduling, monitoring, alerting, and operational runbooks to meet SLAs/SLOs.
• Drive robust data modelling approaches (dimensional, data vault, lakehouse patterns where appropriate), metadata strategy, and performance optimisation for analytical workloads (especially BigQuery). Ensure solutions meet HSBC requirements for data governance, lineage, access control, encryption, auditability, retention, and privacy‑by‑design; partner with Risk/Compliance/InfoSec as needed.
• Establish engineering quality bars including CI/CD, automated testing, code reviews, data quality checks, release governance, and incident/problem management practices.
• Act as a senior technology partner to Product Owners, Architects, Data Analysts/Scientists, and business stakeholders; translate requirements into clear delivery plans and manage expectations. Lead, coach, and develop a team of data engineers (and partner resources), shaping ways of working, capacity planning, and delivery execution in an agile environment.
• Manage dependencies across squads and third parties, ensuring timely delivery, transparent reporting, and effective resolution of blockers, risks, and issues. Drive continuous improvement and FinOps practices—optimising GCP spend (e.g., BigQuery optimisation), improving performance, and standardising best practices for reusability and scalability.
• Define and track key performance indicators (KPIs), including DORA metrics, to measure the effectiveness of engineering practices; report progress and outcomes to senior management and stakeholders.Stay up to date with emerging technologies and industry trends to drive innovation within the engineering practice; mentor and coach team members in application architecture, data management, and AI technologies.
• Accountable with the Head of Architecture to review requirements from a testability perspective and formulate the non‑functional testing strategy; ensure service resilience, sustainability, and recovery time objectives are met for all software solutions delivered. Leverage AI productivity tools (e.g., Microsoft Copilot, Claude, and similar platforms) to improve engineering effectiveness and support delivery outcomes.
  

To be successful in this role, you should meet the following requirements:

• Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred. Significant experience in data engineering and cloud platform delivery, including end‑to‑end solution design and architecture for data platforms and pipelines.
• Strong hands‑on experience with GCP data/platform services (e.g., BigQuery, Dataflow, Pub/Sub, Composer/Airflow, Dataproc, Cloud Storage) and cloud foundations (IAM, networking, encryption/KMS).
• Proven expertise in data modelling and designing performant analytical data structures (e.g., dimensional modelling; data vault/lakehouse patterns beneficial).
• Experience building and operating production‑grade ELT/ETL pipelines with strong observability (monitoring/alerting), operational readiness, and runbook‑driven support.
• Strong understanding of data governance and control requirements in regulated environments (security, privacy‑by‑design, lineage, auditability, retention). Demonstrable experience establishing engineering best practices (CI/CD, automated testing, code quality, release governance) and improving reliability.
• Proven leadership experience managing and developing engineers, including coaching, performance support, and delivery execution in an agile environment.
• Excellent stakeholder management, communication, and interpersonal skills with the ability to influence across a complex organisation.
• Experience working with third‑party vendors/partners and managing delivery dependencies. Exposure to AI concepts and/or participation in AI‑related projects is advantageous.
  

You’ll achieve more when you join HSBC.
www.hsbc.com/careers
HSBC is committed to building a culture where all employees are valued, respected and opinions count. We take pride in providing a workplace that fosters continuous professional development, flexible working and opportunities to grow within an inclusive and diverse environment. Personal data held by the Bank relating to employment applications will be used in accordance with our Privacy Statement, which is available on our website.
Issued by – HSBC Software Development India