Databricks Developer

Infosys; Bengaluru; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Infosys in Bengaluru invites an experienced Databricks Developer to design, build, and optimize scalable data pipelines on Azure Databricks.

You will work with PySpark, Spark SQL, and Delta Lake to transform data, implement ETL/ELT processes, and collaborate with analytics teams to deliver end-to-end solutions in the Azure ecosystem.

This role emphasizes performance tuning, data quality, and leveraging ADLS, Azure Data Factory, and Azure Synapse to drive cloud analytics initiatives.

Qualifications

• Strong hands-on experience with Databricks and Azure Databricks.
• Expertise in PySpark, Spark SQL, and distributed data processing.
• Good understanding of Delta Lake, data lakehouse architecture.
• Experience with Azure ecosystem: Azure Data Factory, ADLS, Azure DevOps (added advantage).
• Solid understanding of ETL/ELT concepts and data modelling.
• Ability to troubleshoot performance issues and optimise code.
• Strong analytical and problem-solving skills.

Responsibilities

• Develop, optimise, and maintain data pipelines using Azure Databricks.
• Work with PySpark/Spark SQL for large-scale data processing.
• Build ETL/ELT workflows integrating various data sources.
• Implement best practices for data quality, performance tuning, and cost optimisation.
• Collaborate with data architects, business stakeholders, and analytics teams to deliver end-to-end solutions.
• Deploy and monitor Databricks workflows and jobs in production environments.
• Manage Delta Lake, notebooks, and data versioning.
• Work with Azure services such as ADLS, ADF, Azure Synapse, etc. (preferred).

Skills

Databricks

Azure Databricks

PySpark

Spark SQL

Delta Lake

ETL/ELT

Data pipelines

Performance tuning

Tools

Azure Data Factory

ADLS

Azure Synapse

Azure DevOps

Job description

We are looking for an experienced Databricks Developer with strong hands‑on expertise in building scalable data pipelines and solutions on Azure Databricks. The ideal candidate should have experience in data engineering, data transformation, and cloud‑based analytics development.

Responsibilities

• Develop, optimise, and maintain data pipelines using Azure Databricks.
• Work with PySpark/Spark SQL for large‑scale data processing.
• Build ETL/ELT workflows integrating various data sources.
• Implement best practices for data quality, performance tuning, and cost optimisation.
• Collaborate with data architects, business stakeholders, and analytics teams to deliver end‑to‑end solutions.
• Deploy and monitor Databricks workflows and jobs in production environments.
• Manage Delta Lake, notebooks, and data versioning.
• Work with Azure services such as ADLS, ADF, Azure Synapse, etc. (preferred).

Requirements

• Strong hands‑on experience with Databricks and Azure Databricks.
• Expertise in PySpark, Spark SQL, and distributed data processing.
• Good understanding of Delta Lake, data lakehouse architecture.
• Experience with Azure ecosystem: Azure Data Factory, ADLS, Azure DevOps (added advantage).
• Solid understanding of ETL/ELT concepts and data modelling.
• Ability to troubleshoot performance issues and optimise code.
• Strong analytical and problem‑solving skills.