Databricks Lakehouse Tech Lead

Tata Consultancy Services Limited; Pleasanton (CA); On-site; USD 130,000 - 140,000; Full time

Benefits offered by this job

Discretionary Incentive

Medical Coverage

Parental Leave

Identity Theft Protection

Training Reimbursement

Vacation & Holidays

401K Plan

Job summary

Tata Consultancy Services Limited in Pleasanton, CA is seeking a Technical Lead to guide data modernization programs. You will assess legacy data platforms, define migration roadmaps to Databricks Lakehouse, and lead cross-functional teams through POC, pilot, and large-scale migrations.

The role requires 10+ years in data engineering and hands-on experience with Databricks, Delta Lake, PySpark, and Azure data services, plus strong governance, CI/CD, and security practices.

Qualifications

• 10+ years of data engineering or data architecture experience.
• 6+ years with Databricks in large-scale environments.
• Expertise in Databricks, Spark, PySpark, SQL, and Delta Lake.
• Experience migrating legacy platforms to Databricks Lakehouse.

Responsibilities

• Lead current-state assessment of data platforms and pipelines.
• Define migration strategy to a Databricks Lakehouse architecture.
• Design target-state data architecture using cloud-native services.
• Lead migration of ETL/analytics workloads to Databricks with PySpark/SQL.
• Mentor teams on Databricks architecture and coding standards.
• Establish CI/CD and governance for data solutions.

Skills

Data engineering

Data architecture

ETL modernization

Enterprise data platform migration

Technical leadership

Migration strategy

Discovery workshops

Architecture reviews

Education

Bachelor of Computer Science

Tools

Databricks

Apache Spark

PySpark

SQL

Delta Lake

Databricks Workflows/Jobs

Alteryx

Azure Databricks

ADLS

Azure Data Factory

Purview

Key Vault

Azure DevOps

Job description

Tata Consultancy Services Limited in Pleasanton, CA is seeking a Technical Lead to guide data modernization programs. You will assess legacy data platforms, define migration roadmaps to Databricks Lakehouse, and lead cross-functional teams through POC, pilot, and large-scale migrations.

The role requires 10+ years in data engineering and hands-on experience with Databricks, Delta Lake, PySpark, and Azure data services, plus strong governance, CI/CD, and security practices.