Data Engineer

Target in India; Bengaluru; On-site; INR 1,800,000 - 4,000,000; Full time

Job summary

Target in India seeks a highly skilled Data Engineer to design, build, and optimize data pipelines in a GCP environment. The role emphasizes Spark pipelines (Scala preferred), BigQuery usage, and Unix scripting to support large-scale data processing.

You will collaborate with data engineering teams to deliver analytics, reporting, and business insights while ensuring data quality, performance, and reliability across workloads.

Qualifications

• 5-9 years of hands-on data engineering experience.
• Proven Spark applications development in Scala.
• Strong GCP experience: Dataproc, BigQuery, Cloud Storage.
• Proficient in SQL, esp. BigQuery SQL.
• Unix/Linux scripting for data automation.
• Git and CI/CD familiarity.

Responsibilities

• Design, develop, and deploy Spark data pipelines.
• Work with GCP components (Dataproc, BigQuery, Cloud Storage).
• Write complex SQL queries in BigQuery.
• Develop Unix shell scripts for automation and orchestration.
• Optimize pipelines for performance, scalability, and reliability.
• Collaborate with data engineers, analysts, and business stakeholders to ensure data quality.
• Contribute to code reviews and CI/CD integration of data workflows.

Skills

Spark (Scala)

BigQuery

Unix shell scripting

SQL

GCP

CI/CD

Git

Data pipelines design

Tools

Dataproc

Cloud Storage

Job description

We are seeking a highly skilled Data Engineer to support the development and optimization of data pipelines within our Google Cloud Platform (GCP) environment. The ideal candidate will have strong experience building Spark pipelines (Scala preferred), working with BigQuery, and writing efficient Unix scripts to support large-scale data processing. You will collaborate with data engineering teams to design, build, and maintain high-performance data pipelines that enable analytics, reporting, and business insights.

Responsibilities:

• Design, develop, and deploy data ingestion and transformation pipelines using Apache Spark (Scala).
• Work within Google Cloud Platform (GCP) components, including Dataproc, BigQuery, and Cloud Storage.
• Write complex SQL queries for data analysis, validation, and transformation in BigQuery.
• Develop Unix shell scripts for automation, data movement, and pipeline orchestration.
• Optimize and troubleshoot data pipelines for performance, scalability, and reliability.
• Collaborate with data engineers, analysts, and business stakeholders to understand requirements and ensure data quality.
• Contribute to code reviews, documentation, and CI/CD integration of data workflows.

Requirements:

• 5-9 years of hands-on experience in data engineering or related roles.
• Proven experience developing Spark applications in Scala.
• Strong experience working in the Google Cloud Platform (GCP) ecosystem, including Dataproc, BigQuery, and Cloud Storage.
• Proficient in SQL (especially BigQuery SQL dialects).
• Strong experience with Unix/Linux scripting for data automation.
• Familiarity with version control (Git) and CI/CD processes.
• Excellent problem-solving and debugging skills.
• Strong communication and documentation abilities.