Lead Data Engineer: Databricks Lakehouse & Modernization

Tata Consultancy Services; Irvine (CA); On-site; USD 110,000 - 140,000; Full time

Job summary

Tata Consultancy Services is seeking a Lead Data Engineer – MSS in Irvine, CA. The role focuses on hands-on Databricks, dbt, and Python, while leading data platform migration and modernization programs.

Ideal candidates bring 8–15 years of data engineering experience, strong leadership, and a proven track record of delivering enterprise-scale data modernization projects across cloud/on‑prem environments.

Qualifications

• 8–15 years of data engineering experience.
• Experience migrating legacy ETL/data warehouses to modern Databricks Lakehouse.
• Hands-on Databricks, dbt, Python and Spark expertise.

Responsibilities

• Design, build, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Python.
• Lead migration of legacy data platforms to Databricks Lakehouse.
• Develop and optimize ELT/ETL processes for large-scale ingestion.
• Establish data quality, monitoring, and CI/CD practices.
• Mentor engineers and drive modernization initiatives.

Skills

Leadership

Mentoring

Communication

Tools

Databricks

dbt

Python

PySpark

SQL

Airflow

Delta Lake

Unity Catalog

Databricks Workflows

Structured Streaming

Medallion Architecture

Job description

Tata Consultancy Services is seeking a Lead Data Engineer – MSS in Irvine, CA. The role focuses on hands-on Databricks, dbt, and Python, while leading data platform migration and modernization programs.

Ideal candidates bring 8–15 years of data engineering experience, strong leadership, and a proven track record of delivering enterprise-scale data modernization projects across cloud/on‑prem environments.