GCP/Streaming/Data Platforms /Consultant Specialist

HSBC; Pune District; On-site; INR 2,800,000 - 4,600,000; Full time

Benefits offered by this job

Flexible working

Professional development

Career growth

Job summary

HSBC Software Development India is seeking an accomplished Data Engineer to join our data platform team in Pune. You will build scalable ingestion pipelines, model data, and drive end-to-end DataOps.

You will work with SQL, Python/Scala/Java, Airflow, and cloud services (BigQuery, Dataproc, Dataflow) to deliver reliable, performant data solutions that power business insights and governance.

Qualifications

• 8–10 years of data engineering experience.
• Strong SQL and Python/Scala/Java for data processing.
• Hands-on with orchestration tools (Airflow) and ETL/ELT patterns.
• Understanding of data warehousing, distributed processing, and performance optimization.
• Experience with data quality, governance, and observability.
• Cloud data platform experience (BigQuery, Dataproc, Dataflow).

Responsibilities

• Build and maintain scalable data ingestion pipelines across diverse sources.
• Develop batch and streaming data processing with CDC patterns.
• Ensure reliable pipeline operations with orchestration, retries, and SLAs.
• Design curated datasets using dimensional or domain-oriented modelling.
• Implement transformation frameworks and optimize query performance.
• Design and operate data lake/warehouse architectures with retention policies.
• Apply DataOps/DevOps practices for CI/CD and IaC.
• Maintain data quality, governance, and observability.

Skills

SQL

Python

Scala/Java

Airflow

Data modeling

Security & governance

Cloud platforms

Education

Bachelor's degree in Computer Science or related field

Tools

BigQuery

Dataproc

Dataflow

Kafka

CDC tooling

ETL/ELT patterns

Job description

Some careers shine brighter than others.

If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.

HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories.

We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions.

We are currently seeking an experienced professional to join our team in the role of Consultant Specalist.

In this role you will be

• Build and maintain scalable data ingestion pipelines across diverse sources (APIs, databases, files, events).
• Develop batch and streaming data processing, including CDC patterns, event-driven pipelines, and near-real-time datasets.
• Ensure reliable pipeline operations through orchestration, retries, backfills, idempotency, and clear SLAs.
• Design and deliver curated datasets/data marts using dimensional or domain-oriented data modelling approaches.
• Implement transformation frameworks and standards (coding conventions, reusable components, documentation) and optimise query performance (e.g., partitioning/clustering).
• Design and operate data lake/warehouse architectures (raw -> refined -> curated), including secure storage, lifecycle policies, and retention management.
• Implement data quality, governance, and observability: checks (completeness/accuracy/timeliness), anomaly detection, reconciliation, lineage, logging/metrics, alerting, cataloguing, access controls, auditability, and PII handling.
• Apply DataOps/DevOps practices: CI/CD for pipelines, Infrastructure as Code where relevant, and operational excellence (incident response, root-cause analysis, continuous improvement).

To be scucessful, you will

• 8-10 years' experience in data engineering, delivering production-grade pipelines and datasets.
• Strong SQL plus proficiency in Python and/or Scala/Java for data processing.
• Hands-on experience with orchestration tools (e.g., Airflow or equivalent) and ETL/ELT patterns.
• Solid understanding of data warehousing concepts, distributed processing, and performance optimisation.
• Experience with data quality/testing frameworks and operational monitoring/alerting.
• Strong security and governance mindset, including access controls, encryption, and PII handling.
• Cloud data platform experience (e.g., BigQuery, Dataproc, Dataflow, or equivalent services).
• Experience with streaming and modern architectures: Kafka/Pub/Sub, CDC tooling, lakehouse patterns, and ML feature pipelines/feature stores.

You'll achieve more when you join HSBC.

HSBC is an equal opportunity employer committed to building a culture where all employees are valued, respected and opinions count. We take pride in providing a workplace that fosters continuous professional development, flexible working and, opportunities to grow within an inclusive and diverse environment. We encourage applications from all suitably qualified persons irrespective of, but not limited to, their gender or genetic information, sexual orientation, ethnicity, religion, social status, medical care leave requirements, political affiliation, people with disabilities, color, national origin, veteran status, etc., We consider all applications based on merit and suitability to the role.

Personal data held by the Bank relating to employment applications will be used in accordance with our Privacy Statement, which is available on our website.

Issued by - HSBC Software Development India.

www.hsbc.com/careers