Senior Data Engineer

Consult Asia; Ghaziabad District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Consult Asia is seeking a data engineer to design, migrate, and deploy data pipelines with Spark and Python/Scala. You will move Hive and Control-M jobs to Airflow/DBT, and integrate with Kafka, Solace, S3, and MinIO, deploying on Kubernetes for scalable, automated, production-grade solutions.

Ideal candidates will have 6–9 years of experience in Spark, Python/Scala, and SQL, with hands-on expertise in Airflow, DBT, Kafka, and data lakehouse tools, and be capable of independent delivery and

Qualifications

• 6–9 years of experience as a Data Engineer with strong Spark, Python/Scala, and SQL skills.
• Hands-on experience with Airflow, DBT, Kafka, Kubernetes, and data lakehouse tools for production-grade pipelines.
• Ability to independently deliver and migrate data pipelines and move Hive/Control-M jobs to Airflow/DBT.

Responsibilities

• Develop and migrate data pipelines using Spark, Python/Scala and SQL.
• Migrate Hive and Control-M jobs to Airflow/DBT and deploy on Kubernetes for scalable operations.
• Integrate with Kafka, Solace, S3, MinIO and ensure production-ready, automated solutions.

Skills

Spark

Python/Scala

SQL

Airflow

DBT

Kafka

Kubernetes

Data Lakehouse

Tools

Hive

Control-M

S3

MinIO

Job description

Required Candidate profile

Develop and migrate data pipelines using Spark, Python/Scala. Move Hive and Control-M jobs to Airflow/DBT. Integrate with Kafka, Solace, S3, MinIO, and deploy on Kubernetes ensuring scalable, automated, production-ready solutions.

Experienced Data Engineer (6–9 yrs) with strong Spark, Python/Scala, and SQL skills. Hands‑on in Airflow, DBT, Kafka, Kubernetes, and data lakehouse tools.Capable of independent delivery and migration