Kubernetes Data Platform

Tata Consultancy Services; Bengaluru Urban; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is seeking an experienced Kubernetes platform engineer to design, build, and operate production-grade platforms. You will deploy Stackable Data Platform services, enable multi-tenant resilience, and drive standardization across teams.

The role emphasizes automation via GitOps, CI/CD improvements, and strong security postures for enterprise-scale data workloads in hybrid cloud environments.

Qualifications

• 5+ years of hands-on Kubernetes platform engineering experience.
• Cluster design, deployment, administration, and troubleshooting.
• Platform upgrades, scaling, and lifecycle management.
• Kubernetes security and multi-tenancy.
• High Availability (HA) and disaster recovery design.

Responsibilities

• Design, build, and maintain production-grade Kubernetes platforms.
• Deploy and operate Stackable Data Platform services on Kubernetes.
• Engineer scalable and resilient multi-tenant platform architectures.
• Develop self-service platform capabilities for data engineering and AI teams.
• Drive platform standardisation and engineering best practices.
• Deploy, configure, and optimise Apache Spark services and workloads.
• Design and support Apache Kafka clusters for streaming workloads.
• Integrate data services with S3-compatible Object Storage platforms.
• Support data lake, analytics, and AI/ML platform use cases.
• Enable workload migration from legacy Hadoop platforms to Kubernetes-native services.
• Implement GitOps-based deployment pipelines using ArgoCD.
• Automate platform provisioning, upgrades, validation, and operational activities.
• Improve CI/CD pipelines and platform release processes.
• Lead troubleshooting of production platform issues.
• Conduct RCA and implement preventative improvements.
• Implement monitoring, observability, alerting, and capacity management practices.
• Drive platform resiliency, availability, backup, and disaster recovery strategies.
• Support platform upgrades, patching, and lifecycle management.
• Implement Kubernetes security best practices.
• Manage RBAC, secrets, certificates, and network policies.
• Support data governance, platform compliance, and operational controls.
• Ensure secure multi-tenant operation across platform services.

Skills

Kubernetes platform

Cluster design

HA & DR design

Kubernetes security

Production troubleshooting

Tools

Stackable Platform

Apache Spark

Apache Kafka

S3-compatible Storage

ArgoCD

Terraform

GitOps

Grafana

Keycloak

Job description

• Design, build, and maintain production-grade Kubernetes platforms.


• Deploy and operate Stackable Data Platform services on Kubernetes.


• Engineer scalable and resilient multi-tenant platform architectures.


• Develop self-service platform capabilities for data engineering and AI teams.


• Drive platform standardisation and engineering best practices.

Data Platform Services

• Deploy, configure, and optimise Apache Spark services and workloads.


• Design and support Apache Kafka clusters for streaming workloads.


• Integrate data services with S3-compatible Object Storage platforms.


• Support data lake, analytics, and AI/ML platform use cases.


• Enable workload migration from legacy Hadoop platforms to Kubernetes-native services.

Automation & DevOps

• Implement GitOps-based deployment pipelines using ArgoCD.


• Automate platform provisioning, upgrades, validation, and operational activities.


• Improve CI/CD pipelines and platform release processes.

Reliability & Operations

• Lead troubleshooting of production platform issues.


• Conduct Root Cause Analysis (RCA) and implement preventative improvements.


• Implement monitoring, observability, alerting, and capacity management practices.


• Drive platform resiliency, availability, backup, and disaster recovery strategies.


• Support platform upgrades, patching, and lifecycle management.

Security & Governance

• Implement Kubernetes security best practices.


• Manage RBAC, secrets, certificates, and network policies.


• Support data governance, platform compliance, and operational controls.


• Ensure secure multi-tenant operation across platform services.

Skills & Experience

Mandatory Skills

• 5+ years of hands-on Kubernetes platform engineering experience.


• Cluster design, deployment, administration, and troubleshooting.


• Platform upgrades, scaling, and lifecycle management.


• Kubernetes security and multi-tenancy.


• High Availability (HA) and disaster recovery design.

Data Platform Technologies

• Strong experience with Stackable Data Platform or comparable Kubernetes-native data platforms.


• Apache Spark (PySpark and/or Scala).


• Spark SQL and performance tuning.


• Apache Kafka administration and operations.


• Real-time and batch data processing architectures.

Storage

• S3-compatible Object Storage (MinIO, Ceph, FlashBlade).


• Data lifecycle management, backup, and recovery.

DevOps & Automation

• ArgoCD.


• Terraform.


• GitOps.


• CI/CD implementation and automation.

Preferred Skills

• Trino.


• NIFI


• Airflow.


• Grafana.


• Vault.


• Keycloak.

Nice to Have

• Telecom or Network Domain experience.


• OpenMetadata.


• DataHub.

Experience Required

• 6+ years of overall platform engineering experience.


• 5+ years operating Kubernetes in production environments.


• Experience supporting mission-critical enterprise data platforms.


• Experience delivering production services in hybrid cloud environments.

What Success Looks Like

The successful candidate will demonstrate the ability to:

• Independently build and operate Kubernetes clusters


• Deploy and manage Stackable platform services


• Scale Spark and Kafka workloads for enterprise use cases


• Design resilient object storage integrations


• Automate platform operations using GitOps and Infrastructure as Code


• Troubleshoot complex platform incidents and perform effective RCA


• Drive platform reliability, security, and operational excellence