T&T - Engineering - Consultant/Senior Consultant | Databricks |Coimbatore

Deloitte & Touche GmbH Wirtschaftsprüfungsgesellschaft; Delhi; Sur place; INR 2 000 000 - 2 800 000; Plein temps

Résumé du poste

Deloitte Touche Tohmatsu India LLP in Coimbatore is seeking a Senior Databricks Engineer to design, build, and support modern Data Lakehouse solutions using Databricks, PySpark, and Delta Lake.

The candidate will contribute to scalable data pipelines, perform performance optimization, and collaborate with data scientists and engineers on enterprise‑grade analytics and AI initiatives.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, IT or related field.
• 4-9 years IT experience with at least 4+ years of hands-on Databricks and Spark development.
• Strong expertise in distributed data processing and Lakehouse architecture.
• Strong knowledge of PySpark, Spark SQL, and performance optimization.
• Experience with Agile development methodologies.
• Excellent analytical, troubleshooting, and communication skills.

Responsabilités

• Design, develop, and maintain scalable data engineering solutions using Databricks.
• Build and optimize batch and real-time data processing pipelines.
• Develop ETL/ELT frameworks using PySpark and Spark SQL.
• Implement Delta Lake architecture and data governance standards.
• Process large-scale structured, semi-structured, and streaming datasets.
• Optimize Spark jobs and cluster performance for maximum efficiency.
• Develop reusable frameworks, notebooks, workflows, and data products.
• Collaborate with business teams, data scientists, analysts, and architects to deliver enterprise-grade solutions.
• Implement monitoring, alerting, security, and compliance requirements.
• Participate in code reviews, deployment activities, and production support.
• Develop technical documentation and mentor junior engineers.

Connaissances

Unity Catalog

Delta Live Tables (DLT)

Data Lake Architecture

Lakehouse Architecture

Data Quality Frameworks

Formation

Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or related field

Description du poste

Select how often (in days) to receive an alert:

Job Title: T&T - Engineering - Consultant/Senior Consultant | Databricks |Coimbatore

T&T - Engineering - Consultant/Senior Consultant | Databricks |Coimbatore

Job requisition ID: 111044

Location: Delhi

Entity: Deloitte Touche Tohmatsu India LLP

The Team

Deloitte’s Technology & Transformation: EAD: Engineering practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning.

Job Title: Senior Databricks Engineer

Experience: 4-9 Years

Location: Coimbatore

Job Summary

We are seeking an experienced Databricks Developer with 6-9 years of expertise in designing, building, and supporting modern Data Lakehouse solutions. The ideal candidate will have strong hands‑on experience in Databricks, PySpark, Delta Lake, cloud platforms, and large‑scale data processing. The role involves building scalable data pipelines, optimizing distributed data workloads, and enabling advanced analytics and AI‑driven initiatives.

Key Responsibilities

• Design, develop, and maintain scalable data engineering solutions using Databricks.
• Build and optimize batch and real‑time data processing pipelines.
• Develop ETL/ELT frameworks using PySpark and Spark SQL.
• Implement Delta Lake architecture and data governance standards.
• Process large‑scale structured, semi‑structured, and streaming datasets.
• Optimize Spark jobs and cluster performance for maximum efficiency.
• Develop reusable frameworks, notebooks, workflows, and data products.
• Collaborate with business teams, data scientists, analysts, and architects to deliver enterprise‑grade solutions.
• Implement monitoring, alerting, security, and compliance requirements.
• Participate in code reviews, deployment activities, and production support.
• Develop technical documentation and mentor junior engineers.

Required Technical Skills

• Unity Catalog
• Delta Live Tables (DLT)
• Data Lake Architecture
• Lakehouse Architecture
• Data Quality Frameworks

Programming Languages

Databases

Streaming & Messaging

DevOps & Tools

Data Engineering Excellence

Collaboration and Leadership

Ownership and Accountability

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or related field.
• 4-9 years of IT experience with at least 4+ years of hands‑on Databricks and Spark development experience.
• Strong expertise in distributed data processing and Lakehouse architecture.
• Strong knowledge of PySpark, Spark SQL, and performance optimization.
• Experience with Agile development methodologies.
• Excellent analytical, troubleshooting, and communication skills.

Preferred Qualifications

• Experience with cloud‑native analytics platforms on Azure, AWS, or GCP.
• Knowledge of Machine Learning workflows within Databricks.
• Experience implementing Delta Lake and Unity Catalog.
• Exposure to Data Governance and Data Security frameworks.
• Relevant Databricks, Azure, or AWS certifications.

Nice to Have

• Delta Live Tables (DLT)
• Unity Catalog
• Generative AI / AI Engineering exposure