Data Engineer - PySpark

Tekskills Inc.; Mumbai; On-site; INR 1,500,000 - 3,000,000; Full time

Job summary

Tekskills Inc. in Mumbai seeks a data engineering specialist with strong Spark (batch + streaming) and Hive expertise. You will build and optimize data pipelines, work with Python/Scala/Java, and collaborate with analytics teams.

The role involves using Airflow/Control-M, DBT, Kafka, and cloud object stores like S3/MinIO, plus deploying with Docker/Kubernetes and handling lakehouse formats Iceberg, Delta Lake, or Hudi.

Qualifications

• Strong expertise in Spark (batch/streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Experience with Airflow or Control-M and DBT.
• Experience with Kafka, Solace, and object stores (S3/MinIO).
• Familiarity with Docker/Kubernetes for deployment.
• Hands-on experience with lakehouse formats (Iceberg, Delta Lake, Hudi).

Responsibilities

• Strong expertise in Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).

Skills

Apache Spark

Hive

Python

Scala

Java

Kafka

Solace

S3

MinIO

Iceberg

Delta Lake

Hudi

Tools

Airflow

Control-M

DBT

Docker

Kubernetes

Job description

Job Summary

Role & responsibilities Strong expertise in Apache Spark (batch + streaming) and Hive. Proficiency in Python, Scala, or Java. Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred). Experience working with Kafka, Solace, and object stores (S3, MinIO). Exposure to Docker/Kubernetes for deployment. Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).

Responsibilities

• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).