Data Engineer / Big Data Engineer / Cloud Data Engineer

Consult Asia; Coimbatore District; On-site; INR 800,000 - 1,200,000; Full time

Job summary

Consult Asia is seeking a data engineer to design, build, and maintain scalable data pipelines using Python, SQL and Spark. You will work with batch and streaming solutions via Kafka and Airflow, and manage cloud data platforms on Azure or AWS.

This role requires hands-on experience with Snowflake and Databricks, plus a track record of delivering reliable analytics data for AI use cases across cloud environments.

Qualifications

• Experience in Python, SQL, Spark, and ETL pipelines.
• Hands-on with Kafka, Airflow, and cloud platforms (Azure/AWS).
• Expertise in Snowflake/Databricks preferred.

Responsibilities

• Design, build, and maintain scalable data pipelines using Python, SQL & Spark.
• Develop batch and streaming solutions with Kafka and Airflow.
• Manage cloud data platforms (Azure/AWS) & ensure high-quality reliable data for analytics and AI use cases.

Skills

Python

SQL

Spark

ETL pipelines

Kafka

Airflow

Cloud platforms

Azure

AWS

Tools

Snowflake

Databricks

Job description

Design, build, and maintain scalable data pipelines using Python, SQL & Spark. Develop batch and streaming solutions with Kafka and Airflow. Manage cloud data platforms (Azure/AWS) & ensure high-quality reliable data for analytics and AI use cases.

Required Candidate profile

Experience in Python, SQL, Spark, and ETL pipelines. Hands-on with Kafka, Airflow, and cloud platforms (Azure/AWS). Expertise in Snowflake/Databricks preferred.