Senior Data Engineer

Consult Asia; Dadri; Presencial; INR 1 500 000 - 2 800 000; Tempo integral

Resumo da oferta

Consult Asia in India is seeking an experienced Data Engineer to design and migrate data pipelines using Spark and Python/Scala, moving Hive and Control-M jobs to Airflow/DBT. You will integrate with Kafka, Solace, S3, and MinIO and deploy on Kubernetes to deliver scalable, automated, production-ready solutions.

The ideal candidate has 6–9 years of hands-on Spark, Python/Scala and SQL experience, with practical work in Airflow, DBT, Kafka and data lakehouse tools, capable of independent delivery

Qualificações

• 6–9 years of experience with Spark, Python/Scala and SQL.
• Hands-on in Airflow, DBT, Kafka, Kubernetes.
• Experience migrating data pipelines and deploying production systems.

Responsabilidades

• Develop and migrate data pipelines using Spark and Python/Scala.
• Move Hive and Control-M jobs to Airflow/DBT.
• Integrate with Kafka, Solace, S3, MinIO and deploy on Kubernetes.

Conhecimentos

Spark

Python/Scala

SQL

Ferramentas

Airflow

DBT

Kafka

Kubernetes

S3/MinIO

Descrição da oferta de emprego

Required Candidate profile

Develop and migrate data pipelines using Spark, Python/Scala. Move Hive and Control-M jobs to Airflow/DBT. Integrate with Kafka, Solace, S3, MinIO, and deploy on Kubernetes ensuring scalable, automated, production-ready solutions.

Experienced Data Engineer (6–9 yrs) with strong Spark, Python/Scala, and SQL skills. Hands‑on in Airflow, DBT, Kafka, Kubernetes, and data lakehouse tools.Capable of independent delivery and migration