Databricks Data Engineer

Tata Consultancy Services; India; On-site; INR 2,000,000 - 4,000,000; Full time

Job summary

Tata Consultancy Services seeks a hands-on Databricks Data Engineer to design, build and optimise scalable data pipelines and Lakehouse solutions for BFSI modernisation programs. You will contribute to migration and AI-ready data platforms across banking and insurance clients.

The role emphasizes PySpark/Spark SQL engineering with Delta Lake, Unity Catalog governance and robust data product delivery, focusing on reliability, performance and scalable architectures.

Qualifications

• Hands-on Databricks Data Engineer to design, build and optimise scalable data pipelines and Lakehouse solutions for large-scale BFSI modernisation.
• Experience with PySpark, Spark SQL, Python and SQL for production-grade data products.
• Strong knowledge of Delta Lake, DLT and Auto Loader for reliable data ingestion and governance.
• Experience with Unity Catalog governance, RBAC, and data lineage to support secure data platforms.

Responsibilities

• Design, build and optimise scalable data pipelines and Lakehouse solutions for BFSI modernisation.
• Deliver production-grade data products across banking, financial services and insurance clients.
• Leverage PySpark/Spark SQL with Delta Lake and Unity Catalog governance for reliability and performance.

Skills

PySpark

Spark SQL

Python

SQL

Delta Lake

Delta Live Tables

Auto Loader

Unity Catalog

RBAC

data governance

data lineage

Lakehouse architecture

ETL/ELT pipelines

performance tuning

Tools

Azure Databricks

AWS Databricks

Job description

Inviting Application for Databricks Data Engineer

Role: Databricks Data Engineer

Exp: 5-10 years

Role & responsibilities

Hands-on Databricks Data Engineer to design, build and optimise scalable data pipelines and Lakehouse solutions for large-scale BFSI modernisation, migration and AI-ready data platform programs. The role combines strong PySpark/Spark SQL engineering with Delta Lake and Unity Catalog governance to deliver reliable, high-performance, production-grade data products across banking, financial services and insurance clients.

Core / Must-Have

• Azure / AWS Databricks (Notebooks, Workflows, Jobs)
• PySpark, Spark SQL, Python, SQL
• Delta Lake, Delta Live Tables (DLT), Auto Loader
• Unity Catalog, RBAC, data governance & lineage
• Medallion / Lakehouse architecture design
• ETL/ELT pipeline engineering & performance tuning