Data Platform Reliability Engineer - L3 Production Support

Tata Consultancy Services Limited; Irvine (CA); On-site; USD 120,000 - 150,000; Full time

Job summary

Tata Consultancy Services Limited is seeking an experienced L3 Production Support Engineer to provide advanced support for enterprise data platforms and critical pipelines. The role emphasizes AWS, Databricks, Airflow, Dremio, Python and SQL with a strong focus on incident management, RCA, stability, and continuous service improvement.

Responsibilities include monitoring workloads on AWS services, troubleshooting incidents, performing RCA, and supporting data quality and metadata validation.

Qualifications

• Experience supporting enterprise data platforms and critical data pipelines.
• Strong expertise in AWS, Databricks, Airflow, Dremio, Python, SQL and data quality frameworks.
• Proven incident management and Root Cause Analysis (RCA) skills; focus on stability and improvements.

Responsibilities

• Provide L3 production support for enterprise data platforms and ensure high availability of critical pipelines.
• Monitor data workloads on AWS (S3, Athena, CloudWatch), Databricks, and Dremio environments.
• Troubleshoot and resolve complex production incidents within SLAs and minimize business impact.
• Perform detailed RCA for recurring issues and implement permanent corrective actions.
• Analyze, debug, and optimize ETL/ELT workflows orchestrated through Airflow.
• Support and maintain Databricks clusters, jobs, notebooks, and runtime configurations.
• Develop and execute SQL and Python scripts for investigation and validation.
• Monitor health using CloudWatch and identify performance bottlenecks and anomalies.
• Manage data residing in AWS S3 and queried via Athena, ensuring reliability and performance.
• Perform data profiling and metadata validation to align with requirements.
• Implement data quality checks, reconciliation processes, and validation frameworks.
• Support schema changes, column additions, and data model enhancements with downtime minimization.
• Execute and validate regression testing during deployments and infrastructure upgrades.
• Manage Databricks runtime upgrades and environment patching.
• Use GitHub for source control, release tracking, and deployment support.
• Collaborate with Data Engineering, Architecture, Business and Infra teams to improve stability.
• Participate in on-call rotations and support incidents during off-hours.

Skills

AWS

S3

Athena

CloudWatch

Airflow

Databricks

Dremio

Python

SQL

GitHub

Metadata

Profiling

Schema

Data quality checks

Regression testing

Runtime upgrades

RCA fixes

Tools

Databricks

Dremio

Airflow

Job description

Tata Consultancy Services Limited is seeking an experienced L3 Production Support Engineer to provide advanced support for enterprise data platforms and critical pipelines. The role emphasizes AWS, Databricks, Airflow, Dremio, Python and SQL with a strong focus on incident management, RCA, stability, and continuous service improvement.

Responsibilities include monitoring workloads on AWS services, troubleshooting incidents, performing RCA, and supporting data quality and metadata validation.