Azure Data Engineer

Rarr Technologies; Bengaluru; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Rarr Technologies in Bengaluru is seeking an experienced Azure Databricks Data Engineer to design, develop, and optimize scalable data solutions.

You will lead data pipelines, collaborate with cross-functional teams, ensure data quality, and mentor peers while driving CI/CD and governance across the data platform.

Qualifications

• Experience designing data pipelines on Azure Databricks.
• Proficiency in ETL/ELT processes and data governance.
• Ability to optimize cluster performance and costs.

Responsibilities

• Design, develop, and lead scalable data pipelines using Azure Databricks.
• Collaborate with data engineers, architects and cross-functional teams.
• Develop and maintain ETL/ELT processes.
• Optimize Databricks cluster performance, jobs, and costs.
• Monitor data processing workflows for quality and reliability.
• Use Databricks notebooks for exploration, analysis, reporting.
• Mentor junior team members on Databricks best practices.
• Ensure compliance with data governance, security, and regulatory standards.
• Participate in architectural discussions and contribute to roadmap.
• Drive automation and CI/CD for Databricks pipelines.
• Stay current with Azure Databricks features and industry best practices.

Skills

Azure Databricks

Data pipelines

Mentoring

Tools

Databricks notebooks

CI/CD tooling

ETL tooling

Job description

We are seeking an experienced Azure Databricks Data Engineer to design, develop, and optimize scalable data solutions. This role leads the implementation of robust data pipelines and integrations, supports data quality and governance, and helps advance the data platform.

Responsibilities

• Design, develop, and lead the implementation of scalable, efficient data pipelines and complex data solutions using Azure Databricks.
• Collaborate with data engineers, architects, and cross-functional teams to understand requirements and deliver robust data integration and transformation solutions.
• Develop and maintain ETL/ELT processes to ensure reliable data flow.
• Optimize Databricks cluster performance, job execution, workload throughput, and costs.
• Monitor data processing workflows to maintain data quality and reliability.
• Use Databricks notebooks for data exploration, analysis, and reporting.
• Mentor and guide junior team members on Databricks best practices and development techniques.
• Ensure compliance with data governance, security, and regulatory standards.
• Participate in architectural discussions and contribute to the data platform roadmap.
• Drive automation and CI/CD processes for Databricks pipelines.
• Stay current with Azure Databricks features and industry best practices.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.