Data Platform Engineering Lead

Tata Consultancy Services; Irvine (CA); On-site; USD 110,000 - 140,000; Full time

Job summary

Tata Consultancy Services in Irvine, CA seeks a Data Platform Engineering Lead to architect, build, and manage enterprise-scale cloud-native data platforms for Asset Management functions. You will oversee platform engineering, DevOps enablement, and reliability across the enterprise data ecosystem.

Responsibilities include defining platform strategy, implementing Lakehouse architectures with Delta, managing Databricks, Terraform, and CI/CD pipelines, and driving SRE practices for performance,

Qualifications

• Expertise in Databricks and AWS Cloud required.
• Strong background in platform engineering and DevOps practices.
• Experience with data platform governance and cost optimization.

Responsibilities

• Lead the design, implementation, and management of enterprise data platforms.
• Define platform engineering strategy aligned with enterprise standards.
• Establish reusable platform frameworks, accelerators, and engineering best practices.
• Drive cloud-native modernization across data and analytics ecosystems.
• Own platform lifecycle management, scalability, availability, and reliability.
• Lead Databricks platform architecture and administration.
• Design Lakehouse architectures leveraging Delta Lake.
• Manage Databricks Workflows, Delta Live Tables, Unity Catalog, MLflow, and security controls.
• Implement CI/CD pipelines and IaC practices with Terraform.
• Enable GitOps and improve deployment velocity and reliability.
• Provide self-service platform capabilities for engineering teams.
• Define platform SLAs/SLOs and monitor observability and incidents.

Skills

Databricks

dbt

AWS Cloud

Terraform

Python

DevOps

Platform Engineering

Data Operations

Tools

GitHub Actions

GitLab CI/CD

Jenkins

Azure DevOps

Terraform

Docker

Kubernetes

Job description

Overview

Data Platform Engineering Lead – We are seeking a highly skilled platform engineering lead to architect, build, and manage enterprise-scale cloud-native data platforms supporting Asset Management business functions. The role will lead platform engineering, infrastructure automation, DevOps enablement, platform reliability, security, and operational excellence across the Enterprise Data Office (EDO).

Key Qualifications

The ideal candidate will have deep expertise in Databricks, dbt, AWS Cloud, Terraform, Python, DevOps, Platform Engineering, and Data Operations, with strong experience supporting investment management organizations.

Responsibilities

1. Lead the design, implementation, and management of enterprise data platforms.
2. Define platform engineering strategy aligned with EDO and enterprise architecture standards.
3. Establish reusable platform frameworks, accelerators, and engineering best practices.
4. Drive cloud-native modernization initiatives across data and analytics ecosystems.
5. Own platform lifecycle management, scalability, availability, and reliability.
6. Lead enterprise-wide Databricks platform architecture and administration.
7. Design and implement Lakehouse architecture leveraging Delta Lake.
8. Manage Databricks Workflows, Delta Live Tables (DLT), Unity Catalog, MLflow, Structured Streaming, Cluster Policies, Security and Access Controls.
9. Optimize platform performance and cost efficiency.
10. Build governance controls across development, test, and production environments.
11. Lead cloud engineering activities using AWS services: S3, ECS, EKS, Lambda, Glue, CloudWatch.
12. Infrastructure as Code (IaC) and build reusable Terraform modules.
13. Automate environment provisioning and implement infrastructure standardization and governance.
14. Maintain infrastructure version control and ensure compliance through IaC practices.
15. Design and implement CI/CD pipelines; automate deployment and release management processes.
16. Enable Infrastructure-as-Code and GitOps methodologies; improve deployment velocity and reliability.
17. Support engineering teams with self-service platform capabilities.
18. Utilize tools such as GitHub Actions, GitLab CI/CD, Jenkins, Azure DevOps, Terraform, Docker, Kubernetes.
19. Support API integrations and operational workflows.
20. Establish Site Reliability Engineering (SRE) practices.
21. Define platform SLAs, SLOs, and operational metrics; lead platform monitoring, observability, and incident management.
22. Drive root cause analysis and preventive actions.

Compensation

Salary Range: $110,000 - $140,000 per year