Databricks Developer at An IT Service Company

Talentzo Delhi; Hyderabad, Pune District, Bengaluru; Hybrid; INR 1,400,000 - 2,200,000; Full time

Job summary

Talentzo Delhi is seeking a data engineer to build and maintain data pipelines using Azure Databricks. You will process large datasets with PySpark and Spark SQL, and develop ETL/ELT workflows from multiple sources to optimize quality, performance and cost.

Responsibilities include managing Delta Lake, notebooks and Databricks workflows while collaborating with data and business teams and leveraging Azure ADLS and ADF services.

Qualifications

• Experience with Azure Databricks and PySpark in large-scale data environments.
• Strong understanding of Delta Lake and Lakehouse concepts.
• Hands-on experience with ETL/ELT design and data modeling.
• Familiarity with Azure data services (ADF, ADLS) and DevOps practices.

Responsibilities

• Build and maintain data pipelines using Azure Databricks.
• Process large datasets with PySpark and Spark SQL.
• Develop ETL/ELT workflows from multiple data sources.
• Optimize data quality, performance, and cost.
• Manage Delta Lake, notebooks, and Databricks workflows.
• Collaborate with data and business teams.
• Work with Azure services like ADLS and ADF.

Skills

Azure Databricks

PySpark

Spark SQL

Delta Lake

Lakehouse architecture

ETL/ELT

Data modeling

Troubleshooting

Performance optimization

Analytical thinking

Tools

Azure Data Factory

Azure Data Lake Storage

Azure DevOps

Databricks notebooks

Job description

Responsibilities

• Build and maintain data pipelines using Azure Databricks.
• Process large datasets with PySpark and Spark SQL.
• Develop ETL/ELT workflows from multiple data sources.
• Optimize data quality, performance, and cost.
• Manage Delta Lake, notebooks, and Databricks workflows.
• Collaborate with data and business teams.
• Work with Azure services like ADLS and ADF.

Preferred candidate profile

• Strong experience with Azure Databricks and Databricks.
• Expertise in PySpark, Spark SQL, and distributed processing.
• Good knowledge of Delta Lake and Lakehouse architecture.
• Experience with Azure Data Factory, ADLS, and Azure DevOps is a plus.
• Strong understanding of ETL/ELT and data modeling.
• Good troubleshooting and performance optimization skills.
• Strong analytical and problem-solving abilities.