Data Engineering Lead

Hackajob; Pune District, Mumbai, Bengaluru; Hybrid; INR 4,000,000 - 7,000,000; Full time

Job summary

Hackajob seeks an experienced Associate Director, Data and Analytics Specialist to own the data platform and end-to-end pipelines. You will provide hands-on technical leadership across GCP services, driving data modelling, governance, and scalable analytics architectures.

You will mentor engineers, manage dependencies, and optimize cloud spend while partnering with product and risk teams. The role emphasizes leadership, agile delivery, and collaboration with stakeholders to meet regulatory and

Qualifications

• Bachelors degree in CS/Engineering; Masters preferred.
• Significant experience in data engineering and cloud platform delivery.

Responsibilities

• Own end-to-end data platform and pipeline architecture (batch/streaming).
• Lead development of resilient ELT/ETL pipelines with orchestration, monitoring, and runbooks.
• Establish CI/CD, automated testing, data quality checks, and release governance.

Skills

GCP Data Services

BigQuery

Dataflow

Pub/Sub

Composer/Airflow

Dataproc

Cloud Storage

IAM & KMS

Data Governance

CI/CD

FinOps

AI Concepts

Education

Bachelor's degree in Computer Science or Engineering

Master's degree preferred

Tools

Airflow

Kubernetes

Cloud Run

Job description

We are currently seeking an experienced professional to join our team in the role of Associate Director, Data and Analytics Specialist

In this role, you will:

• Own end-to-end data platform and pipeline architecture (batch/streaming), defining patterns, standards, and reference designs.
• Provide hands‑on technical leadership across core GCP services such as BigQuery, Cloud Storage, Pub/Sub, Dataflow, Dataproc, Composer (Airflow), Cloud Run/GKE, IAM, Cloud KMS, VPC, and related tooling. Lead development of resilient ELT/ETL pipelines, implementing orchestration, scheduling, monitoring, alerting, and operational runbooks to meet SLAs/SLOs.
• Drive robust data modelling approaches (dimensional, data vault, lakehouse patterns where appropriate), metadata strategy, and performance optimisation for analytical workloads (especially BigQuery). Ensure solutions meet HSBC requirements for data governance, lineage, access control, encryption, auditability, retention, and privacy‑by‑design; partner with Risk/Compliance/InfoSec as needed.
• Establish engineering quality bars including CI/CD, automated testing, code reviews, data quality checks, release governance, and incident/problem management practices.
• Act as a senior technology partner to Product Owners, Architects, Data Analysts/Scientists, and business stakeholders; translate requirements into clear delivery plans and manage expectations. Lead, coach, and develop a team of data engineers (and partner resources), shaping ways of working, capacity planning, and delivery execution in an agile environment.
• Manage dependencies across squads and third parties, ensuring timely delivery, transparent reporting, and effective resolution of blockers, risks, and issues. Drive continuous improvement and FinOps practicesoptimising GCP spend (e.g., BigQuery optimisation), improving performance, and standardising best practices for reusability and scalability.
• Define and track key performance indicators (KPIs), including DORA metrics, to measure the effectiveness of engineering practices; report progress and outcomes to senior management and stakeholders.Stay up to date with emerging technologies and industry trends to drive innovation within the engineering practice; mentor and coach team members in application architecture, data management, and AI technologies.
• Accountable with the Head of Architecture to review requirements from a testability perspective and formulate the non‑functional testing strategy; ensure service resilience, sustainability, and recovery time objectives are met for all software solutions delivered. Leverage AI productivity tools (e.g., Microsoft Copilot, Claude, and similar platforms) to improve engineering effectiveness and support delivery outcomes.

To be successful in this role, you should meet the following requirements:

• Bachelors degree in Computer Science, Engineering, or a related field; Masters degree preferred. Significant experience in data engineering and cloud platform delivery, including end‑to‑end solution design and architecture for data platforms and pipelines.
• Strong hands‑on experience with GCP data/platform services (e.g., BigQuery, Dataflow, Pub/Sub, Composer/Airflow, Dataproc, Cloud Storage) and cloud foundations (IAM, networking, encryption/KMS).
• Proven expertise in data modelling and designing performant analytical data structures (e.g., dimensional modelling; data vault/lakehouse patterns beneficial).
• Experience building and operating production‑grade ELT/ETL pipelines with strong observability (monitoring/alerting), operational readiness, and runbook‑driven support.
• Strong understanding of data governance and control requirements in regulated environments (security, privacy‑by‑design, lineage, auditability, retention). Demonstrable experience establishing engineering best practices (CI/CD, automated testing, code quality, release governance) and improving reliability.
• Proven leadership experience managing and developing engineers, including coaching, performance support, and delivery execution in an agile environment.
• Excellent stakeholder management, communication, and interpersonal skills with the ability to influence across a complex organisation.
• Experience working with third‑party vendors/partners and managing delivery dependencies. Exposure to AI concepts and/or participation in AI‑related projects is advantageous.