Data Platform Lead — Spark, Databricks & Airflow

Tata Consultancy Services; Estado de México; On-site; MXN 1,000,000 - 1,800,000; Full time

Benefits offered by this job

Major Medical Insurance

Life Insurance

Grocery Vouchers

Christmas Bonus

Vacation days according to Mexican劳动法

Profit Sharing (PTU)

Job summary

Tata Consultancy Services is seeking an experienced Data Engineering Tower / Pod Lead to drive enterprise transformation within a global financial services environment. You will lead the Data Engineering Tower and coordinate delivery pods to ensure quality and on-time outcomes.

You will own end-to-end data platform delivery, monitor performance, and collaborate with business and technology leadership to implement best practices, govern data platforms, and optimize throughput.

Qualifications

• Strong experience with Databricks, DBT, PySpark, SQL, Apache Spark, and performance tuning.
• Expertise in Apache Airflow/Astronomer for workflow orchestration and DAG design.
• Deep understanding of Data Engineering, ETL/ELT, and large-scale data platforms.
• Experience with CI/CD, DevOps, Infrastructure as Code (IaC), and automation.
• Strong platform management, monitoring, and incident management experience.
• Excellent leadership and stakeholder management skills.

Responsibilities

• Lead and manage the Data Engineering Tower, overseeing multiple teams and delivery pods.
• Own end-to-end data platform delivery, operational excellence, and SLA adherence.
• Manage cross-functional dependencies, stakeholder communication, and issue escalations.
• Ensure high availability, monitoring, alerting, and production support readiness.
• Collaborate with business, portfolio management, and technology leadership teams.
• Define and implement best practices for platform governance, operational readiness, and performance optimization.

Skills

Databricks

DBT

PySpark

SQL

Apache Spark

Performance tuning

CI/CD

DevOps

IaC

Automation

Data engineering

Monitoring

Incident management

Leadership

Stakeholder management

Airflow

Astronomer

Tools

Airflow

Astronomer

Job description

Tata Consultancy Services is seeking an experienced Data Engineering Tower / Pod Lead to drive enterprise transformation within a global financial services environment. You will lead the Data Engineering Tower and coordinate delivery pods to ensure quality and on-time outcomes.

You will own end-to-end data platform delivery, monitor performance, and collaborate with business and technology leadership to implement best practices, govern data platforms, and optimize throughput.