Lead Data Engineer | Databricks

Sonata Software; Bengaluru; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

Sonata Software in Bengaluru is seeking an experienced Data Hub Lead to guide a small team of developers and business analysts. You will design, plan and execute data solutions, ensuring accuracy and security across datahub initiatives.

You will engage with business stakeholders to translate requirements into scalable data architectures. Responsibilities include leading delivery, creating tickets, and mentoring engineers, while balancing on-prem and cloud-based architectures.

Qualifications

• Minimum 8+ years in Data Engineering, Data Platforms, Analytics or Cloud Data Architecture.
• 3–5+ years hands-on Databricks experience.
• Proven design/implementation of Lakehouse architectures and large-scale data pipelines.
• Strong Azure cloud knowledge incl. data services, storage, networking and security.
• Hands-on with ETL/ELT, data modelling, DataOps, CI/CD and infra automation.

Responsibilities

• Lead a team of 1-2 external developers and guide priorities.
• Engage Global function stakeholders to gather data needs and propose best-in-class solutions.
• Manage project delivery and ensure timely execution.
• Create development tickets and manage backlog of open tickets.
• Assist in designing cloud and on-premise data solutions.

Skills

Stakeholder communication

Team leadership

Project & backlog management

Cloud data architecture

Tools

Databricks

Apache Spark

PySpark

Delta Lake

Unity Catalog

Databricks Workflows

SQL

Azure

Terraform

Jenkins

Kubernetes

Databricks Certification

Job description

Note: need someone who can talk to business stakeholders with experience of Product Owner, as this person will lead a team of developers and business analysts and should design/plan/execute/control the execution

Data Hub Lead - JD

Professional Work Experience

• 8+ years of experience in Data Engineering, Data Platforms, Analytics, or Cloud Data Architecture, with 3–5+ years of hands‑on Databricks experience.
• Strong expertise in Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog, Databricks Workflows, and SQL.
• Proven experience designing and implementing modern Lakehouse architectures, large-scale data pipelines, and real-time data processing solutions.
• Deep knowledge of cloud platforms such as Azure (preferred), including data services, storage, networking, and security.
• Hands‑on experience with ETL/ELT frameworks, data modeling, DataOps, CI/CD pipelines, and infrastructure automation.
• Experience building data products that support BI, Advanced Analytics, Machine Learning, and Generative AI use cases.
• Strong understanding of data governance, metadata management, data quality, lineage, privacy, and compliance frameworks.
• Proven track record leading technical teams, mentoring engineers, and delivering large-scale enterprise data transformation programs.
• Experience with DevOps/DataOps tools such as Azure DevOps, GitHub, Terraform, Jenkins, and Kubernetes is highly desirable.
• Databricks certifications such as Databricks Certified Data Engineer Professional, Databricks Certified Professional Data Architect, or equivalent are preferred.

Strategic Purpose of the Position

Due to expansion and increased complexity of Datahub solutions in the organization, Lead role requires to lead various regional business initiatives and ensuring data accuracy and security within datahub landscape.

Position Description in Brief and Approx Spend of Time in %

Lead team of 1-2 external developers, provide guidance in terms of project and tasks prioritization - 20%

Work with various Global functions stakeholders to gather their data needs and consulting them on best-in-class solutions - 20%

Manage delivery of the projects - 20%

Create development tickets, manage backlog of open tickets and perform - 20%

Understanding technology stack and help in designing of solutions in Cloud and On-prem - 20%