Data Engineer

Tesco; Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Tesco in Bengaluru seeks a skilled Data Engineer (Contractor) to design, build and optimize Spark-based data pipelines in a GCP environment. You will work with Dataproc, BigQuery and Cloud Storage, writing complex BigQuery SQL and Unix scripts for automation.

The role requires 4–6 years in data engineering, strong Spark/Scala experience, and solid SQL/Unix scripting skills. Collaboration with analysts and stakeholders is essential.

Qualifications

• 4–6 years of hands-on data engineering experience.
• Proven Spark applications developed in Scala.
• Strong GCP experience with Dataproc, BigQuery and Cloud Storage.
• Proficient SQL with BigQuery dialects; Unix/Linux scripting.
• Experience with Git and CI/CD; Python is a plus.
• Strong problem solving and documentation skills.

Responsibilities

• Design, develop and deploy Spark-based data ingestion and transformation pipelines.
• Work with GCP components including Dataproc, BigQuery and Cloud Storage.
• Write complex SQL queries for data analysis and validation in BigQuery.
• Develop Unix shell scripts for automation and orchestration.
• Troubleshoot pipelines for performance and reliability; collaborate cross-functionally.
• Contribute to code reviews, documentation and CI/CD integration.

Skills

Spark Scala

BigQuery SQL

Unix scripting

Git CI/CD

Python

Problem solving

Education

Bachelors in CS/Engineering

Tools

Dataproc

BigQuery

Cloud Storage

Git

Job description

We are seeking a highly skilled Data Engineer (Contractor) to support the development and optimisation of data pipelines within our Google Cloud Platform (GCP) environment. The ideal candidate will have strong experience building Spark pipelines (Scala preferred), working with BigQuery, and writing efficient Unix scripts to support large-scale data processing. You will collaborate with data engineering teams to design, build, and maintain high-performance data pipelines that enable analytics, reporting, and business insights.

Responsibilities:

• Design, develop, and deploy data ingestion and transformation pipelines using Apache Spark (Scala).
• Work within Google Cloud Platform (GCP) components including Dataproc, BigQuery & Cloud Storage.
• Write complex SQL queries for data analysis, validation, and transformation in BigQuery.
• Develop Unix shell scripts for automation, data movement, and pipeline orchestration.
• Optimise and troubleshoot data pipelines for performance, scalability, and reliability.
• Collaborate with data engineers, analysts, and business stakeholders to understand requirements and ensure data quality.
• Contribute to code reviews, documentation, and CI/CD integration of data workflows.

Requirements:

• 4-6 years of hands‑on experience in data engineering or related roles.
• Proven experience developing Spark applications in Scala.
• Strong experience working in the Google Cloud Platform (GCP) ecosystem, including Dataproc, BigQuery, and Cloud Storage.
• Proficient in SQL (especially BigQuery SQL dialects).
• Strong experience with Unix/Linux scripting for data automation.
• Familiarity with version control (Git) and CI/CD processes.
• Excellent problem-solving and debugging skills.
• Strong communication and documentation abilities.
• Experience with Python for data processing or automation.
• Knowledge of data governance, data quality, or metadata management best practices.
• Bachelor's degree in Computer Science, Engineering, or a related technical discipline (or equivalent work experience).