Azure Data Engineer (Databricks)

RNR Global Solutions Private Limited; Hyderabad, Gurugram District, Bengaluru; On-site; INR 900,000 - 1,500,000; Full time

Job summary

RNR Global Solutions Private Limited seeks a data engineer to design scalable data pipelines using Spark, SQL, and PySpark on Azure Databricks. The role involves building ETL workflows with Azure Data Factory and maintaining a modern Lakehouse architecture utilizing ADLS and Databricks.

You will collaborate with global Analytics, Data Science and BI teams to deliver data-driven insights, perform data preparation, and support data migrations and production deployments.

Qualifications

• Experience designing scalable data pipelines using Spark, SQL, and PySpark on Azure Databricks.
• Experience building ETL workflows with Azure Data Factory (ADF).
• Experience developing and maintaining a Lakehouse architecture leveraging ADLS and Databricks.
• Data preparation tasks including cleaning, normalization, deduplication, and type transformations.
• Collaboration with DevOps to deploy and manage solutions in production environments.

Responsibilities

• Design and develop scalable data pipelines using Spark, SQL and PySpark on Azure Databricks.
• Build and orchestrate ETL workflows using Azure Data Factory.
• Develop and maintain Lakehouse architecture leveraging ADLS and Databricks.
• Perform data preparation: cleaning, normalization, deduplication, and data type transformations.
• Collaborate with DevOps to deploy and manage production solutions.
• Monitor pipelines, diagnose issues, and implement fixes including RCA.
• Contribute to global Analytics team delivering data-driven insights.
• Collaborate with Data Science and BI teams to share best practices and drive innovation.
• Lead data engineering projects and support cross-functional initiatives.
• Apply change management practices including documentation and training for system upgrades and migrations.

Tools

Spark

SQL

PySpark

Azure Databricks

Azure Data Factory

ADLS

Databricks

Job description

We are hiring for MNC company

Role & responsibilities

• Design and develop scalable data pipelines using Spark SQL and PySpark on Azure Databricks
• Build and orchestrate ETL workflows using Azure Data Factory (ADF)
• Develop and maintain a modern Lakehouse architecture leveraging ADLS and Databricks
• Perform data preparation tasks including data cleaning, normalization, deduplication, and data type transformations
• Collaborate with DevOps teams to deploy and manage solutions in production environments
• Monitor data pipelines, identify issues, and implement corrective actions, including root cause analysis and resolution
• Actively contribute as part of the global Analytics team, delivering data-driven insights and solutions
• Collaborate with Data Science and Business Intelligence teams to share best practices and drive innovation
• Lead data engineering projects and support cross-functional initiatives led by other team members
• Apply change management practices, including documentation, communication, and training, to support system upgrades and data migrations