Lead Data Engineer, Tower – Platform & Pipelines

Tata Consultancy Services Limited; Irvine (CA); On-site; USD 110,000 - 150,000; Full time

Job summary

Tata Consultancy Services Limited in Irvine, CA is seeking an experienced Data Engineer Lead to drive end-to-end data platform delivery, ensure platform stability, and manage multiple data engineering pods within the Asset Management domain.

You will partner with business and technology leadership to design scalable data pipelines, enforce governance, and optimize performance across enterprise-scale data initiatives.

Qualifications

• Extensive experience with Databricks, DBT, Spark, and performance tuning.
• Expertise in workflow orchestration with Airflow/Astronomer and DAG design.
• Deep understanding of Data Engineering, ETL/ELT, and large-scale data platforms.
• Experience with CI/CD, DevOps, IaC, and automation.
• Strong platform management, monitoring, and incident management experience.
• Excellent leadership and stakeholder management skills.

Responsibilities

• Lead and manage the Data Engineering Tower, overseeing multiple teams and pods.
• Own end-to-end data platform delivery, operational excellence, and SLA adherence.
• Drive scalable data pipeline development, data quality, and platform reliability.
• Manage cross-functional dependencies and stakeholder communications.
• Ensure high availability, monitoring, alerts, and production support readiness.
• Collaborate with business and technology leadership to align objectives.
• Define and implement platform governance, operational readiness, and performance optimization.

Skills

Databricks

Apache Spark

Airflow

DBT

CI/CD

IaC

Monitoring

Leadership

Job description

Tata Consultancy Services Limited in Irvine, CA is seeking an experienced Data Engineer Lead to drive end-to-end data platform delivery, ensure platform stability, and manage multiple data engineering pods within the Asset Management domain.

You will partner with business and technology leadership to design scalable data pipelines, enforce governance, and optimize performance across enterprise-scale data initiatives.