Azure Data Engineer

Tata Consultancy Services; India; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

A leading IT service provider in India is seeking an experienced professional to develop data solutions using Azure and Python. The ideal candidate will have 5-10 years of experience and strong skills in SQL, with a solid understanding of Spark architecture. Responsibilities include building data transformations and optimizing performance on big data workloads. Familiarity with Azure Databricks and DevOps practices is essential, making this a great opportunity for someone looking to advance their career in cloud technologies.

Qualifications

• 5-10 years of experience in development of data solutions using cloud platforms.
• Strong programming skills in Python and SQL.
• Experience writing complex SPROCs/Functions/Views using T-SQL.

Responsibilities

• Build complex data transformations on structured and semi-structured data using Pyspark & SQL.
• Tune performance of big data workloads in Spark.
• Utilize Azure Databricks environment for development.

Skills

Python

SQL

Data Solutions Development

Agile

DevOps

Azure Databricks

Pyspark

Tools

Azure DevOps

Git

Job description

Azure + Python + Databricks/Spark+ SQL

At least 5-10 years of experience in development of data solutions using cloud platforms • Strong programming skills in Python • Strong SQL skills and experience writing complex yet efficient SPROCs/Functions/Views using T-SQL • Solid understanding of spark architecture and experience with performance tuning big data workloads in spark • Building complex data transformations on both structure and semi-structured data (XML/JSON) using Pyspark & SQL • Familiarity with Azure Databricks environment • Good understanding of Azure cloud ecosystem; Azure data certification of DP-200/201/203 will be an advantage • Proficient source control using GIT • Good understanding of Agile, DevOps and CI-CD automated deployment (e.g. Azure DevOps, Jenkins)