Databricks Engineer

Talentmatics; Pune District; On-site; INR 2,400,000 - 3,600,000; Full time

Job summary

Talentmatics in Pune is seeking an experienced Senior Data Engineer – Databricks to design, develop, and optimize scalable data platforms using the Databricks Lakehouse. You will build modern data pipelines, implement Delta Lake best practices, and leverage Databricks capabilities.

You will mentor junior engineers and deliver enterprise‑grade data solutions on cloud platforms. The role emphasizes hands‑on development, governance, and collaboration across Azure/AWS/GCP with opportunities to work

Qualifications

• 6–8 years of data engineering experience.
• Strong proficiency in PySpark, Spark SQL, Delta Lake, and Databricks Lakehouse.
• Hands-on expertise in Auto Loader, DLT/SDP, Unity Catalog, and Databricks Workflows.
• Knowledge of dimensional modeling, star/snowflake schemas, data marts, and SCD Type 1/2.
• Experience with AWS, Azure, or GCP cloud storage & security.
• Location: based in Pune or willing to relocate.

Responsibilities

• Design and build production data ingestion pipelines using Auto Loader.
• Implement governance, permissions, and metastore management via Unity Catalog.
• Orchestrate multi-task workflows and complex jobs using Databricks Workflows.
• Perform large-scale data processing using PySpark and Spark SQL.
• Architect and optimize Delta Lake / Delta Tables for ACID and versioning.
• Build or support AI/BI dashboards with Databricks SQL Dashboards, Genie, or similar BI tools.
• Collaborate across Azure, AWS, and GCP cloud storage architectures.

Skills

PySpark

Spark SQL

Delta Lake

Databricks Lakehouse

Auto Loader

Unity Catalog

Databricks Workflows

DLT/SDP

AWS

Azure

GCP

Dimensional modeling

SCD Type 1/2

Tools

Databricks

Databricks SQL Dashboards

Genie

Job description

About the Role

We are seeking an experienced Senior Data Engineer – Databricks to design, develop, and optimize scalable data platforms using the Databricks Lakehouse architecture.

In this role, you will build modern data pipelines, implement Delta Lake best practices, and leverage cutting‑edge Databricks capabilities. You will bring strong hands‑on development skills, mentor junior engineers, and deliver enterprise‑grade data engineering solutions on cloud platforms.

Key Responsibilities

• Design and build production data ingestion pipelines using Auto Loader (cloudFiles, schema inference, and evolution).
• Implement governance, permissions, and metastore management via Unity Catalog.
• Orchestrate multi-task workflows and complex jobs using Databricks Workflows.
• Perform large‑scale data processing using PySpark and Spark SQL.
• Architect and optimize Delta Lake / Delta Tables for ACID transactions, schema enforcement, and versioning.
• Build or support AI/BI dashboards using Databricks SQL Dashboards, Genie, or similar BI tools.
• Collaborate across cloud storage architectures (Azure / AWS / GCP).

What We Are Looking For

• Work Experience: 6 to 8 years in Data Engineering.
• Core Tech Stack: Strong proficiency in PySpark, Spark SQL, Delta Lake, and Databricks Lakehouse architecture.
• Advanced Databricks Features: Hands‑on expertise in Auto Loader, DLT/SDP, Unity Catalog, and Databricks Workflows.
• Data Warehousing: Solid grasp of dimensional modeling, star/snowflake schemas, data marts, and Slowly Changing Dimensions (SCD Type 1/2).
• Cloud Integration: Practical experience with AWS, Azure, or GCP cloud storage & security integrations.
• Location: Must be based in or willing to relocate to Pune.

Why Join Us?

• Work on modern enterprise‑grade data solutions and AI/BI platforms.
• Collaborative engineering environment with clear growth and leadership opportunities.
• Competitive compensation package.