Data Engineer (Hadoop, Python, PySpark)

Tekskills; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tekskills in Bengaluru is seeking a data engineer to design and optimize data pipelines using Spark, Hive, and lakehouse technologies. You will contribute to scalable analytics platforms and collaborate with data teams to deliver reliable data products.

The role requires strong programming skills (Python/Scala/Java) and hands-on experience with Airflow and DBT, plus streaming systems like Kafka and container-based deployment using Docker/Kubernetes.

Qualifications

• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data lakehouse formats (Iceberg, Delta Lake, Hudi).

Skills

Programming: Python/Scala/Java

Big data fundamentals

Tools

Spark

Hive

Airflow

DBT

Kafka

Solace

S3/MinIO

Docker

Kubernetes

Iceberg

Delta Lake

Hudi

Job description

Requirements

• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data lakehouse formats (Iceberg, Delta Lake, Hudi).