Senior Data Engineer

Consult Asia; Surat; On-site; INR 1,200,000 - 2,500,000; Full time

Job summary

Consult Asia wants an Experienced Data Engineer to develop and migrate data pipelines using Spark, Python/Scala, and SQL. The role involves moving Hive and Control-M jobs to Airflow/DBT and integrating with Kafka, S3, MinIO, and deploying on Kubernetes for scalable, production-ready solutions.

The ideal candidate has 6–9 years of hands-on experience with Spark, Python/Scala, SQL, Airflow, DBT, Kafka, and data lakehouse tools, capable of independent delivery and migration.

Qualifications

• 6–9 years of experience as Data Engineer.
• Strong Spark, Python/Scala and SQL skills.
• Hands-on experience with Airflow, DBT, Kafka and Kubernetes.

Responsibilities

• Develop and migrate data pipelines using Spark and Python/Scala.
• Move Hive and Control-M jobs to Airflow/DBT.
• Integrate with Kafka, Solace, S3, MinIO and deploy on Kubernetes.

Skills

Spark

Python

Scala

SQL

Tools

Airflow

DBT

Kafka

Kubernetes

Job description

Required Candidate profile

Develop and migrate data pipelines using Spark, Python/Scala. Move Hive and Control-M jobs to Airflow/DBT. Integrate with Kafka, Solace, S3, MinIO, and deploy on Kubernetes ensuring scalable, automated, production-ready solutions.

Experienced Data Engineer (6–9 yrs) with strong Spark, Python/Scala, and SQL skills. Hands‑on in Airflow, DBT, Kafka, Kubernetes, and data lakehouse tools.Capable of independent delivery and migration