Senior Data Engineer

DXC Luxoft; Pune District; On-site; INR 2,500,000 - 4,200,000; Full time

Job summary

Luxoft is seeking an experienced Data Engineer to design, develop, and support scalable data solutions for surveillance and regulatory reporting use cases. You will onboard new data sources, build ingestion/transform pipelines, implement data quality checks, and support lineage across the platform.

You will work on Dataflow, Pub/Sub, Dataproc, BigQuery, and Cloud Storage to enable efficient management of data across pipelines, with a focus on security, governance, and cost optimization.

Qualifications

• 8+ years of experience in data engineering.
• Strong SQL and data manipulation.
• Deep experience with GCP services (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Dataproc, Cloud Composer).
• Programming in Python, Java, Scala, or Golang.
• Data modeling with star schemas, slowly changing dimensions (SCD).
• Experience with ETL/ELT pipelines and dbt.

Responsibilities

• Data Pipeline Development: batch and real-time pipelines (ETL/ELT).
• Data Warehousing & modeling in BigQuery and data lakes.
• Performance & cost optimization: partitioning, clustering, data lifecycle.
• Data Governance & security: data quality controls, encryption, access management.
• Automation & DevOps: Infrastructure as Code (Terraform) and CI/CD via GitHub.

Skills

SQL

Python

Java/Scala/Golang

Data modeling

Apache Beam

dbt

Education

Bachelor's or Master's degree in Computer Science or Engineering

Tools

Dataflow

Pub/Sub

Dataproc

BigQuery

Cloud Storage

Cloud SQL

Terraform

GitHub CI/CD

Job description

Project description

As part of Client Technology team, the data Engineer will work to design, develop, and support scalable data solutions that underpin strong engineering practices, implicit data controls and the functional requirements for effective surveillance and regulatory reporting use cases. The scope will focus on onboarding new data sources, developing data ingestion and transformation pipelines, implementing data quality controls, supporting lineage and metadata requirements, and enabling efficient management of transactional and non-transactional/ reference data across the platform. The role will support both strategic initiatives and ongoing operational requirements

Responsibilities

• Data Pipeline Development: Design, build, and maintain scalable batch and real-time streaming data pipelines (ETL/ELT) using tools like Dataflow, Pub/Sub, and Dataproc.
• Data Warehousing & Modeling: Architect and optimize enterprise data warehouses, data lakes, and dimensional models inside BigQuery, Cloud Storage, and Cloud SQL.
• Performance & Cost Optimization: Tune query performance, configure proper partitioning and clustering, and manage data lifecycle/retention policies.
• Data Governance & Security: Enforce security controls, encryption, access management, and automated data quality checks.
• Automation & DevOps: Implement Infrastructure as Code (Terraform) and CI/CD pipelines via GitHub or cloud-native deployment tools.

SKILLS

Must have

• Minimum of 8 years of experience in data engineering, with a strong focus on SQL and data manipulation.
• Cloud Platform: Deep experience with GCP services including BigQuery, Dataflow, Pub/Sub, Cloud Storage, Dataproc, and Cloud Composer.
• Programming & Query Languages: Proficiency in Python, SQL, Java, Scala, or Golang.
• Big Data & Analytics Concepts: Data modeling, star schemas, slowly changing dimensions (SCD), Apache Beam, and dbt.
• Education & Experience: Typically a Bachelor's or Master's degree in Computer Science or Engineering,

Nice to have

Banking