Data Engineer - PySpark

Tekskills Inc.; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tekskills Inc. in Bengaluru is seeking a data engineering specialist with deep expertise in Apache Spark (batch and streaming) and Hive. You will leverage Python, Scala or Java, and orchestrate data pipelines using Airflow or Control-M.

Experience with Kafka, Solace, and object stores like S3 or MinIO is essential, along with lakehouse formats such as Iceberg, Delta Lake or Hudi. You will also gain exposure to Docker/Kubernetes for deployment and work with modern data lakehouse architectures in

Qualifications

• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).

Responsibilities

• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).

Skills

Apache Spark

Hive

Python

Scala

Java

Airflow

Control-M

DBT

Kafka

Solace

S3

MinIO

Docker

Kubernetes

Iceberg

Delta Lake

Hudi

Job description

Job Summary

Role & responsibilities Strong expertise in Apache Spark (batch + streaming) and Hive. Proficiency in Python, Scala, or Java. Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred). Experience working with Kafka, Solace, and object stores (S3, MinIO). Exposure to Docker/Kubernetes for deployment. Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).

Responsibilities

• Strong expertise in Apache Spark (batch + streaming) and Hive.
• Proficiency in Python, Scala, or Java.
• Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
• Experience working with Kafka, Solace, and object stores (S3, MinIO).
• Exposure to Docker/Kubernetes for deployment.
• Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).