Azure Data Engineer

Paltech, Inc.; Hyderabad; On-site; INR 1,500,000 - 2,600,000; Full time

Job summary

Paltech, Inc. is seeking a cloud data engineer to build scalable ETL/ELT pipelines that ingest data from APIs, databases and logs into cloud data warehouses. You will optimize performance and costs while leveraging SQL, Python/Scala and modern data orchestration tools.

The role involves maintaining data storage solutions (data lakes and warehouses) and collaborating with cross-functional teams to deliver reliable data solutions for analytics and reporting.

Qualifications

• Bachelor's degree in computer science, IT, or related field.
• Advanced SQL proficiency and Python or Scala.
• Experience with cloud data platforms and data orchestration tools.
• Experience designing and deploying ETL/ELT pipelines in cloud data warehouses.
• Strong communication and teamwork skills.

Responsibilities

• Design and deploy automated ETL/ELT pipelines to ingest data from APIs, databases, and logs into cloud data warehouses.
• Maintain cloud storage solutions (Data Lakes and Data Warehouses).
• Monitor and tune query performance and cloud resource consumption to manage costs.

Skills

Advanced SQL

Python

Scala

Analytical thinking

Team collaboration

Education

Bachelor's degree in Computer Science or IT

Tools

AWS S3/Redshift/Glue

Azure Data Factory/Synapse

GCP BigQuery/Dataflow

Apache Spark

Apache Flink

Kafka

Apache Airflow

dbt

Prefect

Job description

Responsibleforbuildingandmanagingscalablecloud-basedETL/ELTdatapipelines,supportingdataingestion,storage,andanalytics.Theroleinvolvesworkingwithclouddataplatforms,optimizingperformanceandcosts,andleveragingSQL,cloudservices,andmoderndataorchestrationtoolstodeliverreliabledatasolutions.

KeyResponsibilities

• DesignanddeployautomatedETL/ELTpipelinestoingestdatafromvarioussources(APIs,databases,logs)intoclouddatawarehouses.
• Maintaincloudstoragesolutions(DataLakesandDataWarehouses).
• Monitorandtunequeryperformanceandcloudresourceconsumptiontomanagecosts

RequiredSkillsandQualifications

• EducationalBackground:Bachelor’sdegreeincomputerscience,InformationTechnology,orarelatedfield.
• TechnicalSkills:
• CloudPlatforms:AWS(Redshift,S3,Glue)/Azure(DataFactory,Synapse)/GCP(BigQuery,Dataflow).
• Languages:AdvancedSQL(essential)andPythonorScala.
• DataProcessing:ApacheSpark,Flink,orKafkaforreal-timestreaming.
• Orchestration:ApacheAirflow,dbt,orPrefect
• SoftSkills:Excellentcommunication,teamworkabilities,andstronganalyticalandproblem-solvingskills.