Big Data Engineer

Global Infovision; Chennai, Bengaluru; Hybrid; INR 1,500,000 - 2,600,000; Full time

Job summary

Global Infovision in Chennai/Bengaluru is seeking a BigData Engineer to design, develop, and maintain scalable data pipelines using Python, PySpark, and Spark. The role requires 4–10 years of data engineering experience and hands-on work with Hadoop/Hive and SQL.

You will collaborate with DevOps for containerization, CI/CD, and data platform modernization, with exposure to Kubernetes, REST APIs, and GraphQL. Prior experience with Airflow is a bonus.

Qualifications

• 4–10 years of data engineering experience.
• Proficient in Python, Java or Scala.
• Hands-on with Apache Spark or PySpark.
• Strong SQL and data warehousing concepts.
• Experience with Hadoop and Hive.
• Exposure to Kubernetes, CI/CD, REST APIs, and cloud-native data tech.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Python, PySpark, and Spark.
• Work with Big Data technologies, including Hadoop and Hive, for large-scale data processing.
• Develop and optimize SQL queries for data transformation and processing.
• Build and maintain ETL/ELT workflows and data integration pipelines.
• Work with REST APIs and GraphQL for data integration and application connectivity.
• Implement and manage data workflows using Apache Airflow.
• Perform data modelling, data warehousing, and data transformation activities.
• Work with distributed computing frameworks and optimize Spark jobs.
• Collaborate with DevOps teams for deployment, containerization, and CI/CD implementation.
• Ensure data quality, reliability, scalability, and performance across data platforms.

Skills

Python

Java

Scala

SQL

Spark

REST APIs

Kubernetes

Tools

Hadoop

Hive

Airflow

Docker

Kubernetes

CI/CD

GraphQL

Job description

Job Title: BigData Engineer

Job Location: Chennai/Bengaluru

Job Type: Full-Time

Years of experience: 4-10years

Job Description

We are looking for skilled Data Engineers with strong experience in Python, PySpark, SQL, and Big Data technologies to design, develop, and maintain scalable data processing solutions.

The ideal candidate should have hands‑on experience working with distributed data processing frameworks, Hadoop ecosystems, data pipelines, and modern data engineering platforms. Candidates with exposure to Kubernetes, CI/CD, REST APIs, and cloud‑native data technologies will be preferred.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python, PySpark, and Spark.
• Work with Big Data technologies, including Hadoop and Hive, for large-scale data processing.
• Develop and optimize SQL queries for data transformation and processing.
• Build and maintain ETL/ELT workflows and data integration pipelines.
• Work with REST APIs and GraphQL for data integration and application connectivity.
• Implement and manage data workflows using Apache Airflow.
• Perform data modelling, data warehousing, and data transformation activities.
• Work with distributed computing frameworks and optimize Spark jobs.
• Collaborate with DevOps teams for deployment, containerization, and CI/CD implementation.
• Ensure data quality, reliability, scalability, and performance across data platforms.

Mandatory Skills

• Strong programming experience in Python or Java or Scala.
• Hands‑on experience with Apache Spark / PySpark.
• Strong understanding of Big Data technologies Hadoop and Hive.
• Good knowledge of SQL and database concepts.
• Experience in building and maintaining data processing pipelines.
• Understanding of REST API integration.
• Exposure to Kubernetes, Docker, and CI/CD practices.

Good‑to‑Have Skills

• Apache Airflow.
• Apache Iceberg.
• REST APIs and GraphQL.
• Data Warehousing and Data Modelling.
• Kubernetes and CI/CD implementation.
• Apache Ranger and Trino.
• RDBMS concepts.
• Exposure to SAP FSDM / FPSL.
• Experience with distributed data processing and modern Big Data architectures.