Data Engineer with GCP

Luxoft; Bengaluru; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

Luxoft in Bengaluru seeks a senior Data Engineer to design, build and support scalable data solutions for surveillance and regulatory reporting. You will onboard new data sources, develop ingestion/transformation pipelines, and ensure data quality and lineage across the platform.

Responsibilities include batch and real-time pipelines with Dataflow, Pub/Sub, Dataproc; data modeling in BigQuery; governance and security; and IaC with Terraform.

Qualifications

• Minimum of 8 years of experience in data engineering with strong SQL and data manipulation.
• Deep experience with GCP services including BigQuery, Dataflow, Pub/Sub, Cloud Storage, Dataproc, and Cloud Composer.
• Bachelor’s or Master’s degree in Computer Science or Engineering.

Responsibilities

• Data Pipeline Development: Design, build, and maintain scalable batch and real-time streaming data pipelines (ETL/ELT) using tools like Dataflow, Pub/Sub, and Dataproc.
• Data Warehousing & Modeling: Architect and optimize enterprise data warehouses, data lakes, and dimensional models inside BigQuery, Cloud Storage, and Cloud SQL.
• Performance & Cost Optimization: Tune query performance, configure proper partitioning and clustering, and manage data lifecycle/retention policies.
• Data Governance & Security: Enforce security controls, encryption, access management, and automated data quality checks.
• Automation & DevOps: Implement Infrastructure as Code (Terraform) and CI/CD pipelines via GitHub or cloud-native deployment tools.

Skills

SQL

Python

Java

Scala

Golang

Data Modeling

Data Quality

ETL/ELT

Education

Bachelor’s or Master’s degree in Computer Science or Engineering

Tools

BigQuery

Dataflow

Pub/Sub

Dataproc

Cloud Storage

Cloud SQL

Terraform

GitHub

dbt

Apache Beam

Job description

As part of Client Technology team, the data Engineer will work to design, develop, and support scalable data solutions that underpin strong engineering practices, implicit data controls and the functional requirements for effective surveillance and regulatory reporting use cases. The scope will focus on onboarding new data sources, developing data ingestion and transformation pipelines, implementing data quality controls, supporting lineage and metadata requirements, and enabling efficient management of transactional and non-transactional/ reference data across the platform. The role will support both strategic initiatives and ongoing operational requirements

Responsibilities:

• Data Pipeline Development: Design, build, and maintain scalable batch and real-time streaming data pipelines (ETL/ELT) using tools like Dataflow, Pub/Sub, and Dataproc.
• Data Warehousing & Modeling: Architect and optimize enterprise data warehouses, data lakes, and dimensional models inside BigQuery, Cloud Storage, and Cloud SQL.
• Performance & Cost Optimization: Tune query performance, configure proper partitioning and clustering, and manage data lifecycle/retention policies.
• Data Governance & Security: Enforce security controls, encryption, access management, and automated data quality checks.
• Automation & DevOps: Implement Infrastructure as Code (Terraform) and CI/CD pipelines via GitHub or cloud-native deployment tools.

Mandatory Skills Description:

• Minimum of 8 years of experience in data engineering, with a strong focus on SQL and data manipulation.
• Cloud Platform: Deep experience with GCP services including BigQuery, Dataflow, Pub/Sub, Cloud Storage, Dataproc, and Cloud Composer.
• Programming & Query Languages: Proficiency in Python, SQL, Java, Scala, or Golang.
• Big Data & Analytics Concepts: Data modeling, star schemas, slowly changing dimensions (SCD), Apache Beam, and dbt.
• Education & Experience: Typically a Bachelor’s or Master’s degree in Computer Science or Engineering,