Technical Lead

Tata Consultancy Services; Irvine (CA); On-site; USD 120,000 - 200,000; Full time

Job summary

Tata Consultancy Services is seeking a Lead Data Engineer in Irvine to drive data platform migration and modernization using Databricks Lakehouse, dbt, and Python. You will lead a team, shape technical roadmaps, and deliver scalable data pipelines.

The role demands hands-on execution, architecture design, and mentorship to accelerate enterprise data modernization initiatives. Strong communication with analytics teams is essential.

Qualifications

• 8-15 years of data engineering experience in modern data platforms.
• Hands-on experience with Databricks, dbt, and Python required.
• Experience migrating legacy data warehouses and ETL platforms to cloud-native Lakehouse.
• Strong knowledge of ELT/ETL design and data quality frameworks.

Responsibilities

• Design, build, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Python.
• Lead migration of legacy platforms to Databricks Lakehouse and drive modernization efforts.
• Develop and optimize dbt models, tests, and documentation; enable CI/CD for data pipelines.
• Mentor engineers, review code, and enforce engineering standards.
• Collaborate with stakeholders to deliver enterprise-scale data modernization programs.

Skills

Databricks

dbt

Airflow

Python

SQL

Tools

Databricks Lakehouse Platform

Delta Lake

Unity Catalog

Databricks Workflows

Structured Streaming

Job description

Job Description

Lead Data Engineer

Must Have Technical/Functional Skills

Data Bricks , DBT , Airflow ,

Roles & Responsibilities

Lead Data Engineer

Experience

8-15 Years

________________________________________

Role Overview

We are looking for a highly skilled Lead Data Engineer with strong hands-on experience in Databricks, dbt, and Python, and a proven track record of leading and executing data platform migration and modernization programs. The ideal candidate will have experience migrating legacy data warehouses, ETL platforms, and cloud/on-prem data ecosystems to a modern Databricks Lakehouse architecture using dbt-based transformation frameworks. This is a hands-on leadership role requiring deep technical expertise, solution design capabilities, and the ability to mentor engineering teams while driving enterprise-scale data modernization initiatives.

Key Responsibilities

Data Engineering & Development

• Design, build, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Python.
• Develop and optimize ELT/ETL processes for large-scale data ingestion and transformation.
• Implement robust data quality, monitoring, and reconciliation frameworks.
• Build reusable data engineering components and frameworks.

Data Platform Migration & Modernization

• Lead migration of legacy data platforms, data warehouses, and ETL ecosystems to Databricks Lakehouse.
• Transform existing ETL workloads to modern ELT patterns using dbt.
• Analyze source environments and define migration approaches, roadmap, and execution strategy.
• Drive code conversion, performance optimization, and workload modernization efforts.

Databricks Engineering

Develop Solutions Leveraging

• Databricks Lakehouse Platform
• Delta Lake
• Unity Catalog
• Databricks Workflows
• Structured Streaming
• Medallion Architecture (Bronze, Silver, Gold)

Optimize Spark jobs and Databricks workloads for performance and cost efficiency.

dbt Development

• Build and maintain dbt models, macros, tests, and documentation.
• Implement incremental processing and reusable transformation frameworks.
• Establish data lineage, testing, and CI/CD best practices.
• Work closely with business and analytics teams to build trusted datasets.

Leadership & Collaboration

• Lead a team of data engineers and developers.
• Perform code reviews and enforce engineering standards.
• Collaborate with archit ects, produc t owners, business analysts, and stakeholders.
• Mentor junior engineers and drive adoption of best practices.

Salary Range- $120,000-$200,000 a year