Data Engineering Tower Lead – Enterprise Data Platform

Tata Consultancy Services; Irvine (CA); On-site; USD 110,000 - 150,000; Full time

Job summary

Tata Consultancy Services in Irvine seeks an experienced Data Engineer Lead to oversee the Data Engineering Tower, coordinating multiple pods and onshore/nearshore/offshore teams.

You will own end-to-end data platform delivery, drive quality, reliability, and governance, and partner with portfolio management and technology leadership to align with investment objectives.

The role demands deep expertise in Databricks, Spark, Airflow, and IaC, plus strong leadership and stakeholder management skills.

Qualifications

• Strong experience with modern data platforms and large-scale data delivery.
• Expertise in workflow orchestration with Airflow/Astronomer.
• Proficiency in CI/CD, DevOps, and Infrastructure as Code.

Responsibilities

• Lead and manage the Data Engineering Tower, overseeing multiple teams and pods.
• Own end-to-end data platform delivery, operational excellence, and SLA adherence.
• Drive scalable data pipeline development, data quality, and platform reliability.
• Manage cross-functional dependencies, stakeholder communication, and escalations.
• Ensure high availability, monitoring, alerting, and production support readiness.
• Collaborate with business, portfolio management, and technology leadership teams.
• Define and implement best practices for platform governance and performance.

Skills

Databricks

DBT

Apache Spark

Airflow

CI/CD

IaC

Platform monitoring

Leadership

Tools

Astronomer

Terraform

Kubernetes

Job description

Tata Consultancy Services in Irvine seeks an experienced Data Engineer Lead to oversee the Data Engineering Tower, coordinating multiple pods and onshore/nearshore/offshore teams.

You will own end-to-end data platform delivery, drive quality, reliability, and governance, and partner with portfolio management and technology leadership to align with investment objectives.

The role demands deep expertise in Databricks, Spark, Airflow, and IaC, plus strong leadership and stakeholder management skills.