Data Engineer

ITC Infotech; Bengaluru; On-site; INR 3,000,000 - 6,000,000; Full time

Job summary

ITC Infotech in Bengaluru is seeking a highly skilled Data Engineer to design, build, and optimize data pipelines in a Google Cloud Platform (GCP) environment. You will develop Spark-based pipelines (Scala preferred), write SQL for BigQuery, and create Unix scripts to automate data processing at scale, collaborating with analytics and business teams to deliver reliable data insights.

The role emphasizes performance, scalability, and robust CI/CD integration to maintain data quality across

Qualifications

• 46 years of hands-on experience in data engineering.
• Proven experience developing Spark applications in Scala.
• Strong experience with Google Cloud Platform (Dataproc, BigQuery, Cloud Storage).
• Proficient in SQL (BigQuery dialects).
• Unix/Linux scripting for data automation.
• Experience with Git and CI/CD processes.
• Excellent problem-solving and debugging skills.
• Strong communication and documentation abilities.

Responsibilities

• Design, develop, and deploy data ingestion and transformation pipelines using Apache Spark (Scala).
• Work within GCP components including Dataproc, BigQuery, Cloud Storage.
• Write complex SQL queries for data analysis and transformation in BigQuery.
• Develop Unix shell scripts for automation, data movement, and orchestration.
• Optimize and troubleshoot data pipelines for performance and scalability.
• Collaborate with data engineers, analysts, and stakeholders to ensure data quality.
• Contribute to code reviews, documentation, and CI/CD integration of data workflows.

Job description

Role & responsibilities

JOB DESCRIPTION

We are seeking a highly skilled Data Engineer to support the development and optimization of data pipelines within our Google Cloud Platform (GCP) environment. The ideal candidate will have strong experience building Spark pipelines (Scala preferred), working with BigQuery, and writing efficient Unix scripts to support large-scale data processing.
You will collaborate with data engineering teams to design, build, and maintain high-performance data pipelines that enable analytics, reporting, and business insights.

Responsibilities

• Design, develop, and deploy data ingestion and transformation pipelines using Apache Spark (Scala).
• Work within Google Cloud Platform (GCP) components including Dataproc, BigQuery & Cloud Storage.
• Write complex SQL queries for data analysis, validation, and transformation in BigQuery.
• Develop Unix shell scripts for automation, data movement, and pipeline orchestration.
• Optimize and troubleshoot data pipelines for performance, scalability, and reliability.
• Collaborate with data engineers, analysts, and business stakeholders to understand requirements and ensure data quality.
• Contribute to code reviews, documentation, and CI/CD integration of data workflows.

Required Skills & Qualifications

• 46 years of hands-on experience in data engineering or related roles.
• Proven experience developing Spark applications in Scala.
• Strong experience working in Google Cloud Platform (GCP) ecosystem — including Dataproc, BigQuery, Cloud Storage.
• Proficient in SQL (especially BigQuery SQL dialects).
• Strong experience with Unix/Linux scripting for data automation.
• Familiarity with version control (Git) and CI/CD processes.
• Excellent problem-solving and debugging skills.
• Strong communication and documentation abilities.

Nice-to-Have Skills

• Experience with Python for data processing or automation.
• Knowledge of data governance, data quality, or metadata management best practices.

Education

• Bachelor’s degree in Computer Science, Engineering, or a related technical discipline (or equivalent work experience).