Lead Databricks Developer

GyanSys Inc.; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

GyanSys Inc. in Bengaluru seeks a Lead Databricks Development to design, develop, and implement scalable data engineering solutions using Databricks, ensuring reliable pipelines for enterprise manufacturing use cases.

You will lead a team of data engineers, review code, optimize SQL queries, and collaborate with business stakeholders to translate requirements into technical tasks while maintaining data quality and performance.

Qualifications

• Experience leading data engineering initiatives.
• Strong SQL expertise with complex queries.
• Hands-on Databricks, PySpark, Delta Lake, ETL/ELT concepts.
• Ability to translate business problems into technical requirements.
• Experience with semiconductor/manufacturing data is a plus.
• Excellent communication with senior stakeholders.

Responsibilities

• Lead design, development, and implementation of scalable data pipelines.
• Provide technical leadership to a team of data engineers and perform reviews.
• Optimize and troubleshoot complex SQL queries and data transformations.
• Collaborate with business and technical stakeholders to translate requirements.

Skills

Databricks

SQL

PySpark

Delta Lake

Data Engineering

Leadership

Stakeholder Management

Tools

Databricks

PySpark

Delta Lake

Job description

• Lead Databricks Development: Lead the design, development, and implementation of scalable data engineering solutions using Databricks, ensuring high-quality and reliable data pipelines for enterprise and semiconductor/manufacturing use cases.
• Technical Leadership: Provide technical leadership to a team of data engineers, guide day-to-day development activities, conduct code/design reviews, resolve technical challenges, and ensure adherence to engineering best practices.
• Strong SQL Expertise: Demonstrate strong hands-on expertise in SQL, including complex joins, CTEs, subqueries, window functions, aggregations, data transformation, performance tuning, and troubleshooting of complex queries.
• Databricks & Data Engineering: Strong working knowledge of Databricks, PySpark, Delta Lake, notebooks, workflows/jobs, data transformation, and ETL/ELT concepts. Should be comfortable remaining hands-on while leading the team.
• Client & Stakeholder Management: Act as a key technical point of contact for client stakeholders. Independently participate in discussions with business and technical teams, understand their requirements, ask the right questions, and provide clear technical perspectives and solutions.
• Requirement Gathering & Translation: Work closely with business stakeholders to understand data requirements and translate business problems into technical requirements, data models, transformation logic, and actionable development tasks for the engineering team.
• Communication & Confidence: Excellent verbal and written communication skills with the confidence to lead discussions with senior client stakeholders. Should be able to explain complex technical concepts in a simple and business-friendly manner and confidently challenge or clarify requirements when required.
• Semiconductor / Manufacturing Data: Experience working with semiconductor, manufacturing, supply chain, equipment, production, quality, or other industrial data is preferred. Exposure to high-volume, complex, and time-series manufacturing datasets will be an advantage.
• Data Quality & Performance: Ensure data quality, reliability, scalability, and performance across pipelines. Troubleshoot data issues, identify root causes, and work with relevant teams to implement sustainable solutions.
• Ownership & Delivery: Take end-to-end ownership of assigned data engineering initiatives, including requirement clarification, technical design, task allocation, development oversight, stakeholder communication, delivery tracking, and production support. Should be capable of managing multiple priorities and coordinating a sizeable engineering team.