Data Engineer -Databricks platform, DLT

The Glove; Bengaluru; On-site; INR 900,000 - 1,500,000; Full time

Job summary

The Glove is seeking an experienced Data Engineer in Bengaluru, India to design and maintain robust data pipelines on the Databricks platform. You will implement Lakeflow, Delta Lake, Unity Catalog, and Auto Loader to support scalable data architecture and streaming solutions.

Ideal candidates have 8–12 years of data engineering experience, strong Python/SQL/PySpark skills, and a passion for building state‑of‑the‑art data platforms with real‑time capabilities.

Qualifications

• 8–12 years of Data Engineering experience with a focus on big data pipelines.
• Proficiency in Python, SQL and PySpark.
• Strong knowledge of Structured Streaming and the Databricks platform.
• Experience with Delta Lake, Unity Catalog, Auto Loader, and streaming tables.

Responsibilities

• Design, develop, and maintain robust data pipelines using Databricks and DLT.
• Implement and manage Lakeflow pipelines, Delta Lake, Unity Catalog, and Auto Loader.
• Develop and optimize streaming tables and ensure seamless data flow across platforms.
• Collaborate with cross‑functional teams to understand data requirements and deliver efficient solutions.
• Monitor performance and troubleshoot issues to ensure reliable data processing.

Skills

Data engineering

Python

SQL

PySpark

Structured Streaming

Tools

Databricks platform

Delta Lake

Unity Catalog

Auto Loader

Lakeflow

Job description

About the Company

We are an innovative leader in the tech industry, dedicated to delivering cutting-edge solutions and fostering a collaborative environment where talent thrives. Our focus is on leveraging technology to drive efficiency and create value for our clients. We believe in investing in our people and cultivating a diverse and inclusive workplace that encourages growth and innovation.

About the Role

We are seeking a highly skilled Data Engineer with expertise in the Databricks platform and Data Lake technologies to join our dynamic team in Bangalore, India. This permanent position is an exciting opportunity for a professional with significant experience in data engineering and a passion for implementing advanced data solutions. If you have a strong foundation in Python, SQL, PySpark, and Structured Streaming, and are eager to work with state-of-the-art technology, we want to hear from you.

Key Responsibilities

• Design, develop, and maintain robust data pipelines using the Databricks platform and DLT.
• Implement and manage Lakeflow pipelines, Delta Lake, Unity Catalog, and Auto Loader to support data architecture.
• Develop and optimize streaming tables and ensure seamless data flow across platforms.
• Collaborate with cross-functional teams to understand data requirements and deliver efficient solutions.
• Monitor performance and troubleshoot issues to ensure optimal data processing and reliability.

Requirements

• Proven experience as a Data Engineer with 8 to 12 years in the field.
• Proficiency in Python, SQL, and PySpark.
• Strong knowledge of Structured Streaming and Databricks platform.
• Experience with Delta Lake, Unity Catalog, Auto Loader, and streaming tables.
• Excellent problem-solving skills and attention to detail.
• Ability to work effectively in a team-oriented, collaborative environment.

Preferred Qualifications

• Prior experience in a similar role within the tech industry.
• Familiarity with Lakeflow pipeline management.
• Strong communication skills and the ability to convey technical concepts to non-technical stakeholders.

Location

Bangalore, India