Data Engineer (Hadoop, Python, PySpark)

Tekskills; Chennai District; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Tekskills is seeking a data engineering specialist with strong expertise in Apache Spark (batch and streaming) and Hive. The role requires proficiency in Python, Scala, or Java and hands-on experience with orchestration tools such as Airflow, as well as SQL transformation frameworks like DBT.

Ideal candidates will have exposure to Kafka, Solace, and cloud object stores (S3, MinIO), plus Docker/Kubernetes deployments and data lakehouse formats including Iceberg, Delta Lake, or Hudi.

Qualifications

• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data lakehouse formats (Iceberg, Delta Lake, Hudi).

Skills

Apache Spark

Hive

Python

Scala

Java

Airflow

Control-M

SQL

DBT

Kafka

Solace

S3

MinIO

Iceberg

Delta Lake

Hudi

Tools

Docker

Kubernetes

Job description

Requirements

• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data lakehouse formats (Iceberg, Delta Lake, Hudi).