Senior Open-Source Data Platform Architect

Tata Consultancy Services Limited; New York (NY); On-site; USD 140,000 - 180,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage

Family Support: Leaves

Insurance Options: 401K & More

Certification & Training Reimbursement

Job summary

Tata Consultancy Services Limited in New York City area seeks a senior Architect with 14+ years of enterprise data experience. You will design end-to-end, open-source data platforms for batch, streaming, analytics, and AI workloads, lead modernization efforts, and ensure secure, scalable, and observable environments.

Role requires strong leadership, stakeholder management, and hands-on expertise with Spark, Kafka, and data lakehouse technologies. Travel may be required for client engagements.

Qualifications

• 14+ years of enterprise data architecture and data engineering experience.
• Experience across data to insights end-to-end value chain.
• Hands-on with Spark, PySpark, Kafka, Flink, Airflow, NiFi, Trino.
• Designing open-source data platform architectures and modern Lakehouse solutions.
• Experience with batch, streaming, and real-time processing.
• Knowledge of data governance, security, RBAC, and privacy controls.
• Proficiency with Iceberg, Hudi, Delta Lake architectures.
• Experience with Kubernetes and CI/CD in DataOps.

Responsibilities

• Engage with stakeholders to understand data strategy, platform requirements, architecture goals, and transformation priorities.
• Define end-to-end architecture for enterprise open-source data platforms supporting batch, streaming, analytics, and AI workloads.
• Lead modernization of data lakes, lakehouse platforms, and distributed processing frameworks.
• Architect scalable, secure, highly available data platforms using open-source technologies.
• Define data ingestion, integration, streaming, storage, and consumption patterns with Spark, Kafka, Flink, NiFi, Trino.
• Establish data governance, metadata, lineage, privacy, and observability standards across the platform.

Skills

Enterprise Data Architecture

Open-Source Data Platforms

Distributed Data Processing

Data Engineering

Apache Spark

PySpark

Kafka

Flink

Airflow

NiFi

Trino

SQL

Python

Scala

Java

Data Lakehouse

DataOps

CI/CD

Observability

Metadata Governance

Kubernetes

RBAC & Security

Cloud Migration

Education

Bachelor's degree in Computer Science / Engineering / Information Systems

Cloud Data Engineering Certification

Kubernetes Certification

Tools

PostgreSQL

Cassandra

MongoDB

OpenSearch

Job description

Tata Consultancy Services Limited in New York City area seeks a senior Architect with 14+ years of enterprise data experience. You will design end-to-end, open-source data platforms for batch, streaming, analytics, and AI workloads, lead modernization efforts, and ensure secure, scalable, and observable environments.

Role requires strong leadership, stakeholder management, and hands-on expertise with Spark, Kafka, and data lakehouse technologies. Travel may be required for client engagements.