Azure Data Engineer

PalTech; Hyderabad; On-site; INR 1,200,000 - 2,200,000; Full time

Job summary

PalTech in Hyderabad is seeking a data engineer to design and manage scalable cloud-based ETL/ELT pipelines, enabling reliable ingestion, storage, and analytics. You will optimize performance and costs across AWS, Azure, and GCP while leveraging SQL, Python/Scala, and modern orchestration tools to deliver robust data solutions.

You will build pipelines from APIs, databases, and logs, maintain data lakes and data warehouses, and monitor query performance and resource usage to control cloud costs.

Qualifications

• Bachelor’s degree in computer science, information technology, or a related field.
• Advanced SQL (essential) and Python or Scala.
• Data processing with Spark, Flink, or Kafka for real-time streaming.
• Orchestration with Apache Airflow, dbt, or Prefect.
• Soft skills: excellent communication, teamwork, and strong analytical and problem-solving skills.

Responsibilities

• Design and deploy automated ETL/ELT pipelines to ingest data from APIs, databases, and logs into cloud data warehouses.
• Maintain cloud storage solutions (Data Lakes and Data Warehouses).
• Monitor and tune query performance and cloud resource consumption to manage costs.

Skills

SQL

Python

Scala

Communication

Teamwork

Analytical Thinking

Problem Solving

Education

Bachelor’s degree in Computer Science or Information Technology

Tools

AWS Redshift

S3

Glue

Azure Data Factory

Azure Synapse

GCP BigQuery

Dataflow

Airflow

dbt

Prefect

Apache Spark

Flink

Kafka

Job description

Role Summary

Responsible for building and managing scalable cloud-based ETL/ELT data pipelines, supporting data ingestion, storage, and analytics. The role involves working with cloud data platforms, optimizing performance and costs, and leveraging SQL, cloud services, and modern data orchestration tools to deliver reliable data solutions.

Key Responsibilities

• Design and deploy automated ETL/ELT pipelines to ingest data from various sources (APIs, databases, logs) into cloud data warehouses.
• Maintain cloud storage solutions (Data Lakes and Data Warehouses).
• Monitor and tune query performance and cloud resource consumption to manage costs

Required Skills And Qualifications

Educational Background: Bachelor’s degree in computer science, Information Technology, or a related field.

• Cloud Platforms: AWS (Redshift, S3, Glue) / Azure (Data Factory, Synapse) / GCP (BigQuery, Dataflow).
• Languages: Advanced SQL (essential) and Python or Scala.
• Data Processing: Apache Spark, Flink, or Kafka for real-time streaming.
• Orchestration: Apache Airflow, dbt, or Prefect
• Soft Skills: Excellent communication, teamwork abilities, and strong analytical and problem-solving skills.