Azure Databricks Developer

United States Digital Space LLC; Maharashtra; On-site; INR 1,870,000 - 2,530,000; Full time

Job summary

United States Digital Space LLC is seeking an experienced Azure Databricks Developer to design and optimize scalable data solutions on the Azure platform. The role focuses on building data pipelines, leveraging Spark and PySpark, and integrating with ADLS and data warehouses.

The ideal candidate will have strong Python/Scala skills, deep knowledge of the Azure data ecosystem, and hands-on experience with Databricks notebooks, jobs, and CI/CD pipelines.

Qualifications

• Experience with Azure Databricks and Spark-based data processing.
• Proficient in Python and/ or Scala for data engineering tasks.
• Strong understanding of the Microsoft Azure ecosystem and ADLS data lake usage.
• Expertise in ETL/ELT pipelines and data transformation.
• Familiarity with Databricks notebooks, jobs, and CI/CD workflows.

Responsibilities

• Design scalable data pipelines using Azure Databricks.
• Develop Spark/PySpark based data processing solutions.
• Integrate data across Microsoft Azure services and ADLS.
• Build and optimize ETL/ELT pipelines for large datasets.
• Work with data security, governance, and access controls.

Skills

Azure Databricks

Apache Spark

PySpark

Python

Scala

SQL

ADLS

CI/CD

Tools

Databricks notebooks

Databricks jobs

Cluster management

Job description

Position: Azure Databricks Developer

Experience: 6–12 Years

Locations: Bangalore | Chennai | Mumbai

Employment Type: Full-Time

Role Overview

We are looking for an experienced Azure Databricks Developer with strong expertise in data engineering, cloud technologies, and large-scale data processing. The ideal candidate will be responsible for designing, developing, and optimizing scalable data solutions using Azure Databricks and Microsoft Azure.

Key Responsibilities

• Design and develop scalable data pipelines using Azure Databricks.
• Develop data processing solutions using Apache Spark / PySpark.
• Build and integrate data solutions across Microsoft Azure services.
• Develop and optimize ETL/ELT pipelines for large datasets.
• Work with Azure Data Lake Storage (ADLS) and data warehouse solutions.
• Perform data transformation, cleansing, validation, and optimization.
• Integrate data from multiple sources including databases, APIs, and files.
• Optimize Spark jobs, Databricks notebooks, clusters, and SQL queries.
• Implement data security, access controls, and governance best practices.
• Work with CI/CD pipelines for Databricks deployment and automation.
• Collaborate with Data Engineers, Architects, Analysts, and business teams.

Required Technical Skills

• Strong hands-on experience with Azure Databricks.
• Strong knowledge of Apache Spark \& PySpark.
• Strong experience with Python and/or Scala.
• Good understanding of Microsoft Azure ecosystem.
• Experience with Azure Data Lake Storage (ADLS).
• Strong knowledge of SQL \& data processing.
• Hands-on experience in ETL/ELT and data pipeline development.
• Experience with Databricks notebooks, jobs, workflows, and cluster management.
• Knowledge of performance tuning and optimization in Spark/Databricks.
• Understanding of Azure security, authentication, and access management.
• Exposure to Azure DevOps / CI-CD practices.

Good to Have

• Experience with Delta Lake / Delta Live Tables.
• Knowledge of Azure Data Factory (ADF).
• Experience with Databricks SQL.
• Knowledge of Unity Catalog and data governance.
• Exposure to DevOps, Git, and infrastructure automation.
• Experience working on large-scale enterprise data platforms.

Preferred Candidate Profile

• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.
• Ability to work independently and deliver scalable data solutions.
• Strong understanding of modern Cloud Data Engineering practices.

Pay: From ₹2,200,000.00 per year

Work Location: In person