Data engineer (Apache Hadoop, Python and PySpark)

Tekskills Inc.; Kolkata District; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Tekskills Inc. in Kolkata seeks an experienced data engineer to design scalable data pipelines using Spark (batch and streaming) and Hive. You will collaborate across teams to implement robust data solutions and ensure data quality.

You will work with Python/Scala/Java, orchestrate with Airflow and Control-M, and implement DBT transformations, Kafka, Solace, and lakehouse formats such as Iceberg, Delta Lake, and Hudi, with containerised deployment using Docker/Kubernetes.

Qualifications

• 4 to 10 years of hands-on data engineering experience.
• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).

Skills

Data engineering

Apache Spark

Hive

Python

Scala

Java

Airflow

Control-M

SQL

DBT

Kafka

Solace

S3

MinIO

Docker

Kubernetes

Iceberg

Delta Lake

Hudi

Tools

Airflow

Control-M

DBT

Kafka

Solace

Docker

Kubernetes

Iceberg

Delta Lake

Hudi

S3

MinIO

Job description

• 4 to 10 years of hands-on data engineering experience.
• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).