Databricks Developer

Kumaran Systems; Chennai District; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Kumaran Systems is seeking an experienced Senior Databricks Developer with 5+ years of experience in designing and implementing scalable data engineering solutions.

You will design, build, and maintain robust data pipelines on Databricks, optimize Spark workloads, and develop ETL/ELT workflows for structured and unstructured data.

Collaborate with data scientists, analysts, and business stakeholders to enable data-driven decisions, ensure data governance, and document architectures and processes.

Qualifications

• 5+ years of experience in data engineering or big data development.
• Hands-on Databricks experience.
• Distributed data processing with Apache Spark.
• Proficiency in Python, Scala, and SQL.
• Experience building scalable ETL/ELT data pipelines.
• Experience with data lakes or data warehouse environments.
• Experience with cloud platforms (Azure, AWS, or GCP).
• Strong knowledge of data modeling and data integration concepts.
• Excellent analytical, troubleshooting, and performance optimization skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Build and optimize large-scale data processing solutions using Apache Spark.
• Develop and manage ETL/ELT workflows for ingesting, transforming, and processing data.
• Write efficient data processing scripts using Python, Scala, and SQL.
• Implement and manage data lake and data warehouse architectures.
• Optimize performance of data pipelines and Spark jobs.
• Integrate Databricks with cloud data services and storage platforms.
• Collaborate with data scientists, analysts, and engineering teams to deliver reliable data solutions.
• Implement best practices for data governance and data quality.

Skills

Databricks

Apache Spark

Python

Scala

SQL

ETL/ELT pipelines

Data modeling

Cloud platforms

Job description

About Us

Kumaran Systems: Engineering Future-Ready Transformations Since 1992, Kumaran Systems has been helping enterprises bridge the past and the future, reimagining legacy systems and embracing AI-led possibilities. With over 34 years of delivery excellence, we specialise in re-engineering core applications, driving cloud transformation, and building automation-first, GenAI-enabled ecosystems across the Banking and Financial Services, Insurance, Telecom, and Automotive sectors. We help organisations not just upgrade their systems but reimagine what’s possible.

About Us

Kumaran Systems: Engineering Future-Ready Transformations Since 1992, Kumaran Systems has been helping enterprises bridge the past and the future, reimagining legacy systems and embracing AI-led possibilities. With over 34 years of delivery excellence, we specialise in re-engineering core applications, driving cloud transformation, and building automation-first, GenAI-enabled ecosystems across the Banking and Financial Services, Insurance, Telecom, and Automotive sectors. We help organisations not just upgrade their systems but reimagine what’s possible.

Why Kumaran?

We blend engineering discipline with AI innovation, helping clients modernise with confidence, automate with clarity, and scale with purpose. Our global delivery model ensures agility, responsiveness, and seamless collaboration, with clients always at the heart of every engagement. At Kumaran, we don’t just solve problems, we engineer future-ready transformations.

Job Summary

We are seeking an experienced Senior Databricks Developer with 5+ years of experience in designing and implementing scalable data engineering solutions

The role involves building robust data pipelines, optimizing large-scale data processing workloads, and collaborating with data engineers, analysts, and business stakeholders to enable data-driven decision-making.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks .
• Build and optimize large-scale data processing solutions using Apache Spark .
• Develop and manage ETL/ELT workflows for ingesting, transforming, and processing structured and unstructured data.
• Write efficient data processing scripts using Python , Scala , and SQL .
• Implement and manage data lake and data warehouse architectures.
• Optimize performance of data pipelines and Spark jobs.
• Integrate Databricks with cloud data services and storage platforms.
• Collaborate with data scientists, analysts, and engineering teams to deliver reliable data solutions.
• Implement best practices for data governance, data quality, and security.
• Troubleshoot data pipeline issues and ensure system reliability and scalability.
• Document data architecture, workflows, and operational processes.

Required Skills & Qualifications

• 5+ years of experience in data engineering or big data development.
• Strong hands‑on experience with Databricks .
• Extensive experience with distributed data processing using Apache Spark .
• Proficiency in Python , Scala , and SQL .
• Strong experience building scalable ETL/ELT data pipelines.
• Experience working with large datasets in data lake or data warehouse environments.
• Experience with cloud platforms such as Microsoft Azure , Amazon Web Services , or Google Cloud Platform .
• Strong knowledge of data modeling, data architecture, and data integration concepts.
• Excellent analytical, troubleshooting, and performance optimization skills.