Data Engineer

Insight Global; Bengaluru; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

Insight Global in Bengaluru is seeking a highly hands-on Core Data Engineer with strong software engineering fundamentals to build scalable data solutions on Google Cloud Platform. You will develop and maintain Spark jobs, improve data pipelines, and optimize performance across large datasets.

The role requires 5+ years in data engineering and backend software, with strong skills in Scala/Java/Python, Spark, Kafka, BigQuery, and advanced SQL. Experience with GCP and Airflow is a plus.

Qualifications

• 5+ years of experience in Data Engineering and Backend Software Engineering.
• Strong programming expertise in Scala (preferred), Java, or Python.
• Deep hands-on experience developing and optimizing Apache Spark applications.
• Experience building and maintaining Kafka-based data pipelines.
• Strong experience working with BigQuery and large-scale analytical datasets.
• Advanced SQL skills with experience tuning complex queries.

Responsibilities

• Develop and maintain Spark jobs and data pipelines.
• Optimize large-scale data processing systems and business logic.
• Integrate with Kafka and BigQuery to improve data platform performance.
• Build high-performance data applications on GCP.

Skills

Scala/Java/Python

Apache Spark

Kafka

BigQuery

SQL

Data engineering

Backend software engineering

Tools

Apache Spark

Kafka

BigQuery

Airflow

Google Cloud Platform

Job description

JOB DESCRIPTION

We are seeking a highly hands-on Core Data Engineer with strong software engineering fundamentals and deep expertise in building scalable data solutions. This is not a traditional ETL-focused role. The ideal candidate will have a strong coding background and experience developing business logic, optimizing large-scale data processing systems, and building high-performance data applications on GCP. You will be responsible for developing and maintaining Spark jobs, enhancing business logic within data pipelines, integrating with Kafka and BigQuery, and improving the performance and scalability of our data platform.

REQUIRED SKILLS AND EXPERIENCE

5+ years of experience in Data Engineering and Backend Software Engineering Strong programming expertise in Scala (preferred), Java, or Python Deep hands‑on experience developing and optimizing Apache Spark applications Experience building and maintaining Kafka-based data pipelines Strong experience working with BigQuery and large-scale analytical datasets Advanced SQL skills with experience tuning complex queries

NICE TO HAVE SKILLS AND EXPERIENCE

Experience with Apache Airflow Experience working within Google Cloud Platform (GCP)