Databricks Engineer

NextGen Digital Solutions - NDS; Maharashtra; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

NDS Infotech Ltd is seeking an experienced Databricks Engineer to design, develop and maintain scalable data pipelines. You will work with Azure Databricks, Spark, Python, and SQL to process large datasets and optimize ETL/ELT workflows.

The role emphasizes collaboration with analysts and engineers to ensure secure, reliable data solutions. 3–5 years of hands-on data engineering experience is required, with knowledge of ADLS Gen2, Delta Lake and Delta Tables.

Qualifications

• 3–5 years of hands-on experience in Data Engineering.
• Strong expertise in Azure Databricks, PySpark and SQL.
• Experience building and optimising ETL/ELT pipelines.
• Familiarity with ADLS Gen2, Delta Lake and Delta Tables.
• Understanding of data warehousing concepts and data modelling.
• Experience with Git and CI/CD practices.

Responsibilities

• Design, develop and maintain scalable data pipelines using Azure Databricks.
• Work with Apache Spark and PySpark to process and transform large datasets.
• Develop and optimise ETL/ELT workflows for efficient data processing.
• Work with Azure Data Lake Storage and other cloud-based data services.
• Implement data transformations, data quality checks and validation processes.
• Troubleshoot performance issues and optimise Spark jobs and queries.
• Collaborate with data analysts, architects and other engineering teams.
• Ensure data security, reliability and best practices across data engineering solutions.

Skills

Python

PySpark

SQL

Apache Spark

Data pipelines

Tools

Azure Databricks

Delta Lake

Delta Tables

Azure Data Lake Storage

Git

CI/CD

Azure Data Factory

Databricks Workflows

Unity Catalog

Synapse Analytics

Microsoft Fabric

Job description

Company Description

NDS Infotech Ltd is a Microsoft Solution Partner specializing in consulting, implementation, and support services for digital transformation with Automation, Low Code, Analytics & AI. NDS offers solutions in Intelligent Applications, Agentic AI solutions, Custom AI Assistants, Data & Analytics, Robotic/Business Process Automation, Low Code Solutions, and Demand, Project & Portfolio Management across various industries.

Job Description – Databricks Engineer

Location: Mumbai Experience: 3–5 Years Employment Type: Full-time

Company Description

NDS Infotech Ltd is a Microsoft Solution Partner specializing in consulting, implementation, and support services for digital transformation with Automation, Low Code, Analytics & AI. NDS offers solutions in Intelligent Applications, Agentic AI solutions, Custom AI Assistants, Data & Analytics, Robotic/Business Process Automation, Low Code Solutions, and Demand, Project & Portfolio Management across various industries.

Job Overview

We are looking for an experienced Databricks Engineer with 3–5 years of hands-on experience in designing, developing and maintaining scalable data engineering solutions. The ideal candidate should have strong expertise in Databricks, Apache Spark, Python and SQL, along with experience in building and optimising data pipelines.

Key Responsibilities

• Design, develop and maintain scalable data pipelines using Azure Databricks.
• Work with Apache Spark and PySpark to process and transform large datasets.
• Develop and optimise ETL/ELT workflows for efficient data processing.
• Work with Azure Data Lake Storage and other cloud-based data services.
• Implement data transformations, data quality checks and validation processes.
• Troubleshoot performance issues and optimise Spark jobs and queries.
• Collaborate with data analysts, architects and other engineering teams.
• Ensure data security, reliability and best practices across data engineering solutions.

Required Skills & Experience

• 3–5 years of hands-on experience in Data Engineering.
• Strong experience with Azure Databricks.
• Proficiency in Python, PySpark and SQL.
• Good understanding of Apache Spark architecture and distributed data processing.
• Experience in developing ETL/ELT pipelines.
• Knowledge of Azure Data Lake Storage (ADLS Gen2).
• Experience working with Delta Lake and Delta Tables.
• Understanding of data warehousing concepts and data modelling.
• Familiarity with Git and CI/CD practices.

Good to Have

• Experience with Azure Data Factory.
• Knowledge of Microsoft Fabric and Synapse Analytics.
• Experience with Databricks Workflows and Unity Catalog.
• Exposure to cloud-based data architecture and optimisation.

Preferred Candidate

• Strong analytical and problem-solving skills.
• Ability to work independently and collaboratively.
• Good communication and coordination skills.

Skills: data,azure,spark