Databricks Lakehouse Lead Engineer

Tata Consultancy Services; Irvine (CA); On-site; USD 120,000 - 200,000; Full time

Job summary

Tata Consultancy Services in Irvine, CA seeks a Lead Data Engineer to design, build, and optimize scalable data pipelines using Databricks, PySpark, SQL, and Python, and to lead modernization programs migrating legacy data warehouses to a Databricks Lakehouse.

This hands-on leadership role requires deep technical expertise, solution design capabilities, and the ability to mentor engineering teams while driving enterprise-scale data modernization initiatives.

Qualifications

• 8–15 years of data engineering experience.
• Hands-on with Databricks, dbt, and Python.
• Experience migrating legacy data warehouses to modern lakehouse architectures.
• Experience leading engineering teams and delivering enterprise-scale data modernization.

Responsibilities

• Design, build, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Python.
• Develop and optimize ELT/ETL processes for large-scale data ingestion and transformation.
• Implement data quality, monitoring, and reconciliation frameworks.
• Lead migration of legacy data platforms to Databricks Lakehouse.

Skills

Databricks

dbt

Airflow

Python

SQL

PySpark

Tools

Databricks Lakehouse Platform

Delta Lake

Unity Catalog

Databricks Workflows

Structured Streaming

Medallion Architecture

Job description

Tata Consultancy Services in Irvine, CA seeks a Lead Data Engineer to design, build, and optimize scalable data pipelines using Databricks, PySpark, SQL, and Python, and to lead modernization programs migrating legacy data warehouses to a Databricks Lakehouse.

This hands-on leadership role requires deep technical expertise, solution design capabilities, and the ability to mentor engineering teams while driving enterprise-scale data modernization initiatives.