Azure Databrick Lead

Hexaware Technologies; Hyderabad; Hybrid; INR 2,000,000 - 4,000,000; Full time

Job summary

Hexaware Technologies in Hyderabad seeks a senior data engineer to design, build, and maintain large-scale data pipelines using Azure Databricks.

You will collaborate with cross-functional teams, develop PySpark on Spark clusters, write complex SQL, and integrate Azure Data Factory, Synapse Analytics, and Logic Apps to deliver robust data solutions.

Qualifications

• 6–11 years of hands-on experience in designing and developing big data solutions.
• Expertise with Azure Databricks and PySpark.
• Strong SQL query optimization and troubleshooting skills.
• Experience with ML model implementation using PySpark is a plus.

Responsibilities

• Design, develop, and maintain large-scale data pipelines using Azure Databricks.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions.
• Develop complex SQL queries to optimize database performance and troubleshoot issues.
• Implement machine learning models using PySpark on Spark clusters.
• Ensure seamless integration of Azure services such as Data Factory, Synapse Analytics, and Logic Apps.

Skills

Azure Databricks

PySpark

SQL

Machine learning

Tools

Azure Data Factory

Synapse Analytics

Logic Apps

Job description

Roles and Responsibilities

• Design, develop, and maintain large-scale data pipelines using Azure Databricks.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions.
• Develop complex SQL queries to optimize database performance and troubleshoot issues.
• Implement machine learning models using PySpark on Spark clusters.
• Ensure seamless integration of Azure services such as Data Factory, Synapse Analytics, and Logic Apps.

Desired Candidate Profile

• 6-11 years of experience in designing and developing big data solutions using Azure Databricks.
• Strong expertise in working with PySpark on Spark clusters.
• Proficiency in writing complex SQL queries for query optimization and troubleshooting purposes.
• Experience with implementing machine learning models using PySpark is an added advantage.