Lead Data Engineer - Databricks Lakehouse Modernization

Tata Consultancy Services; Irvine (CA); On-site; USD 110,000 - 140,000; Full time

Job summary

Tata Consultancy Services in Irvine, CA is seeking a Lead Data Engineer for Information Management, with hands-on Databricks, dbt, and Python, to lead enterprise-scale data modernization initiatives.

You will design, build, and optimize scalable data pipelines, migrate legacy data platforms to a Databricks Lakehouse, implement CI/CD for data workflows, and mentor a team of engineers to deliver trusted datasets.

Qualifications

• 8–15 years of data engineering experience.
• Experience leading data platform migration and modernization programs.
• Hands-on experience with Databricks, dbt, and Python in large-scale environments.

Responsibilities

• Design, build, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Python.
• Lead migration of legacy data platforms to Databricks Lakehouse and modern ELT patterns with dbt.
• Develop data quality, monitoring, and CI/CD for data workflows; mentor engineers.

Skills

Databricks

dbt

Python

PySpark

SQL

Airflow

Tools

Databricks Lakehouse

Delta Lake

Unity Catalog

Databricks Workflows

Structured Streaming

Medallion Architecture

Job description

Tata Consultancy Services in Irvine, CA is seeking a Lead Data Engineer for Information Management, with hands-on Databricks, dbt, and Python, to lead enterprise-scale data modernization initiatives.

You will design, build, and optimize scalable data pipelines, migrate legacy data platforms to a Databricks Lakehouse, implement CI/CD for data workflows, and mentor a team of engineers to deliver trusted datasets.