Big Data Engineer

Ambiquest Hyderabad; Chennai, Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Ambiquest Hyderabad in Chennai/Bengaluru is seeking an experienced Big Data Engineer to design, build and maintain scalable data pipelines using Python, Spark/PySpark, Hadoop and Hive.

You will optimize Spark SQL workloads, work with REST APIs, GraphQL, and container orchestration via Kubernetes, while integrating data platforms with CI/CD pipelines and Iceberg-based data lake solutions.

Qualifications

• Strong programming experience in Python.
• Hands-on experience with Apache Spark and PySpark.
• Strong knowledge of Hadoop and Hive.
• Experience with Spark SQL.
• Knowledge of REST APIs and GraphQL.
• Experience with Kubernetes and CI/CD.
• A Apache Iceberg experience is an added advantage.

Responsibilities

• Design, develop, and maintain scalable Big Data processing solutions.
• Build and optimize distributed data processing pipelines using Python and Apache Spark/PySpark.
• Work extensively with the Hadoop ecosystem, including HDFS and Hive.
• Develop and optimize Spark SQL queries for large-scale data processing.
• Integrate data platforms with REST APIs and GraphQL services.
• Work with Kubernetes for container orchestration and deployment.
• Implement and maintain CI/CD pipelines.
• Work with Apache Iceberg for modern data lake and table management solutions.

Skills

Python

Apache Spark/PySpark

Hadoop

Hive

Spark SQL

REST APIs & GraphQL

Kubernetes

CI/CD

Apache Iceberg

Tools

Kubernetes

CI/CD

REST APIs

GraphQL

Iceberg

Job description

We're Hiring: Big Data Engineer

We are looking for an experienced Big Data Engineer with strong hands-on expertise in Python, Apache Spark/PySpark, Hadoop, and Hive to build and maintain scalable big data processing solutions.

Location: Chennai/Bengaluru
Experience: 4-10
Notice Period: Upto 45 Days

Key Responsibilities:

• Design, develop, and maintain scalable Big Data processing solutions.
• Build and optimize distributed data processing pipelines using Python and Apache Spark/PySpark.
• Work extensively with the Hadoop ecosystem, including HDFS and Hive.
• Develop and optimize Spark SQL queries for large-scale data processing.
• Integrate data platforms with REST APIs and GraphQL services.
• Work with Kubernetes for container orchestration and deployment.
• Implement and maintain CI/CD pipelines.
• Work with Apache Iceberg for modern data lake and table management solutions.

Required Technical Skills:

• Strong programming experience in Python.
• Hands-on experience with Apache Spark and PySpark.
• Strong knowledge of Hadoop and Hive.
• Experience with Spark SQL.
• Knowledge of REST APIs and GraphQL.
• Experience with Kubernetes and CI/CD.
• Apache Iceberg experience is an added advantage / required for relevant positions.

IMPORTANT STRICT REQUIREMENT

WE ARE LOOKING ONLY FOR CORE BIG DATA ENGINEERS.

AWS Data Engineers – NOT REQUIRED
Azure Data Engineers – NOT REQUIRED
GCP Data Engineers – NOT REQUIRED